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ABSTRACT 

The Small House program at Slauson Junior High 
School, Ann Arbor, Michigan, during the 1971-72 school year vas 
evaluated. Two purposes of the evaluation study were to assess 
developments in the Small House program and to develop a prototype or 
model for evaluating innovative educational programs that delineates 
the evaluation, design, and analytic techniques. Small House was a 
pilot program that otudents could elect as an alternative to 
traditional education. It was designed to humanely alter the 
administrative and psychological environment of public junior high 
schools through flexible modular scheduling and team teaching. This 
study evaluated goals that Small House teachers set for themselves: 
normal studert achievement in reading and mathematics, positive 
student attitudes toward school, and humane behavior in both students 
and teachers. A control group consisted of traditional mathematics 
classes at , the junior high school. The California Reading Test, 
California Arithmetic Test, School Sentiment Index, and an attitude 
questionnaire were administered and student and teacher behaviors 
were observed. Evidence suggested that the program goals were 
achieved. However, variables such as administrative support, 
selection bias, team-cluster structure, and program newness 
confounded the evidence. Development of an evaluation model for 
innovative educational programs was not met due to methodological 
flaws and limitations in the study. However, a skeletal model was 
suggested that contains instructional, institutional, and behavioral 
dimensions. (ND) 



Documents acquired by ERIC inchide many informal unpublished materials not available from other sources. ERIC makes every 
effort to obtain the best copy available. Nevertheless, items of marginal reproducibility are often encountered and this affects the 
quality of the microfiche and hardcopy reproductions ERIC makes available via the ERIC Document Reproduction Service (EDRS). 
v.^ is not responsible for the quality of the original document. Reproductions supplied by EDRS are the best that can be made from 



U$ OCPAMTMCNT or HEALTH. 
COUCATlONtWELFAllE 
NATIONAL iNSTiTUTI OP 
iOUCATION 

»MtS DOCUMENT HAS BEEN REPRO- 
OUCEO EXACTLY AS RECEIVED FROfV 
THE PERSON OR ORGANrZATrON ORtGlN. 
ATtNOiT POINTS OF VIEW OR OPfNiONS 
STATEO 00 NOT NECESSARILY REPRE- 
SENT OFFICIAL NATiCNAl INSTITUTE OF 
EDUCATION POSITION OR POLICY 



A STUDY OF ATTITUDE CHANGE AND 
ACHIEVEMENT IN AN INTIOVATIVE 
PROGRAM AT THE JUNIOR HIGH SCHOOL LEVEL 

by 

C. Lawrence Dugan 



" . . . secondary schools tend to be even more 
authoritarian and repressive than elementary 
schools • • • . ANT THE JUNIOR HIGH SCHOOL, BY 
ALMOST UNANIMOUS AGREEMENT, IS THE WASTELAND — 
ONE IS TEMPTED TO SAY CESSPOOL — OF AMERICAN 
EDUCATION (Silberman, Crisis in the Classroom) 



June 1973 



vow EWORD 

Opcr;it:ion Sninll House uvis initinted in Septcmhcr, 1971 at Slnuson 
Junior IMoU Sc^-ool , ,\nn Arbor, Mic^it^nn. This oxpcr imentn 1 pro^>rnm 
Wc'is a direct: result' of i^c Puinancncs s in h'ducnfiot^ report subiintr.cd 
to t^c Ann Arl)or Ronrd of Lduccition on January 13,' 1971 and represen- 
ted a concer-ced iffort to alter t^e administrative and psychological 
>^abits of public Junior hi school education. Many of t^e goals and 
guidelines t-'or Snmll House are set fort^ in the Huinaneness Reportu 

This report: constitutes an attempt to evaluate t^e Small I'ouse 
program conducted at Slauson between September 1971 and June 1972. 
Just; as Small House was a pilot progrri t., tf^is was intended to be a 
pilot study with a two-fold purpose: 

ao To assess developments in t^e on-going Small "ouse program 

b. To develop a prototype or model for evaluating educational 
programs, particularly innovative programs, w^ic^- would de- 
lineate t^c p^ssential components of evaluation, as well as 
design considerations and analytic techniques. 

At thf. outset, it was ^oped t^at we could develop, pre-test, and 
implement t^ese assessment techniques aU within one year. We either 
underestimated t^e complenity and enormity of t^e task confronting us 
or we overestimated our capabilities .rvj resources, or both. In any 
event, while the results of this stud^ ^ight be interesting and in 
some cases, provocative, much of what - .r. evolved remains to be fina- 
lized and validated in the future, "ov.-var, T believe we were success- 
ful in developing the skeleton of a broad-based multi-method model 
for evaluation w^m ch would be applicable to other educational programs. 
This portotype is by no means considered a finished product, merely a 
guide for future research. 

There are several other important considerations which should be 
mentioned at this time: 

a. Parts of the research suffer from methodological flaws and 

limitation-, which effectively preclude reaching any definitive 
conclusions, promising though the results may seemc The nature 
of these limitations will be specified in the appropriate sec- 
tions; I wish to stress that the results should be read with 
these limitations in mindo 

bo There is another study of Small House being conducted by Marcia 
Radin and Mary Ann Morris at the Univer5:l- of Michigan xchich 
covers areas ignored or only touch<» ' o in this report. I feel 
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t'-nt hot'- of thoHc reports nro criticMl to 3 full cv.iluntion 
.•mi.] uiult rstnnclinj- of Sit;i11 llnusc, 

c. THcM-c is no adrqiKitc^ dcscr i pr i ,v,i of t'n- experimental trciUmcnt 
( (Snnll Pousc propram) nvnil.iMc ;incl T 1 1 not presu.r.r to present 
one v>crc. There is n vicl-. ot.ipe tiinclc v.'hich attempts to summarize 
tv-e proc-rnni .-md v^-ic'^ could Ik- obtnincd by cnntacthg Mr. Paul 
Sinno at Slauson Junior Hipt^ Sc^^ool . From t»-e start, t '^e Small 
Hous. Team bas stressed t^at tl^is program vas t'^eir own and, in 
f^at sense, unique. A brief skctc»^ of Small House been pro- 

vided in f-e Introduction and attempts x-jere niade in ot>.cr sections 
to identify some c>^aracteristics of t>^e Small House program. 

T owe Ji special fcv^ank you to Ken T'eile w^-o persevered wi me from 
tVe very start of t^is undertaking to c^e bittersweet end and was essen- 
tial to its completion. T also wis>^ to acknowledc^e my sincerest appre- 
ciation to Anne Savis»^insky for ».er >^elp in s'-aping our observational 
sc^^enies and to Professors Morse, Nadelman, and Dyer and t»^e members of t^^e 
l>>ursday morning seminar for t>^eir comments and suggestions. To t^^e 
tcac>^ers and students at Slauson, T am particularly grateful; despite 
t»^c constant demands made on t»^em, t>^ey remained patient and cooperative 
t»-roug»^out. 

T»-is report will be organized into seven major sections as follows: 

C»>aptrr T - Introduction - Includes a brief history and background of 
Small House, a very brief description of t^e program and goals f-at 
guided t'^e researc*^. 

Chapter IT - Psyc'-ometric Evaluation - Summarizes t^e results of t^^e 
testing conducted. Eac*- test is '-andled in a separate subsection. 

Oapter III - Questionnaire - Describes t^e admi ni strat i cn and results of 
a questionnaire specifically designed for t^e Small House evaluation. 

C>^apter IV - Systematic Observations - Summarizes t^e results of one 
mont^ of observations at Slauson, in Small House and in tV^e Control groups 
wit^^ respect to gross distinctions among teaC-er and student, be^avio rs. 

adapter V - Nonsystemati c Observations - Includes anecdotal materials 
from notes compiled over t^e year nonsystemat i ca 1 ly t>^at bear on t^^e 
eva luat i on. 

C»>apter VI - In Retrospect - Synthesizing t^^e information and the 
development of t^c infrastructure of an evaluation model. 

C^'apter VII - Conclusions and comments. 
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CHAPTKR ] 



TNTKODirTlON 



A. SMALL )K)Vr>\^i A BKTFK 0VI:RVTK\' 



T^c iiiipcUis for l^c Sto 1 1 in>usc concept st.ctmied from t^e ^^imnneno ss 
report sulMi.itted rr. t^c Ann Arbor Hocircl of Kducntion; at t^at t\ir,c, or 
soon thorcaftrr, srvcntK ^',rnclr toacKrs in eac^ junior ^ ip,^ school vtrc 
solicil(cl for volunteers to i,i.r,iM a Str/,11 i^ousG the following September. 
Approxi:natel y 70-75 teachers froii Slauson vo 1 unteered , f rom xc'm c^ four 
vero selected — alon^ Xvit^- one counselor — to act as t^e Spin H '^ouso 
team. T^rsc five persons Xv^ere ^;ivrn sotc planning time over t^e surnmer 
to develop a prooram consistent x,;it^ t^-e c^nls set fort^ in the Humaneness 
report; in addition, tbcy established a s' t of goals for t^eniselveso 
P-ese individuals shaped t^e pror^rarn t^at xcas to be implemented at Slauson 
in September. TKerc was no oHicial Policy and Procedure statement nor 
any formal espoused philosophy for t^e Small T^ousc program, except for 
t^e general guide linrs of the ^^imaneness report and t^e goals which the 
teachers set for themselves; U-ese latter established the objectives of 
this report. 

One of the direct results of this unspecified philosophy xcas that the 
details of the operation of Small Vouse were worked out tentatively, and 
continued to evolve over the course of the year, within the framework 
provided. Hoicever, those directly concerned with the program made it 
perfectly clear that Small House was not intended to be an Open Classroom 
or a Free School. (As will be seen later, that distinction, like the 
distinction b( tween graded ani nongradcd classes, can be a nebulous onej 
This tentativeness (highly reninisccnt of the espoused p^ilosophy of 
Frazier in JCalden Two) makes any but the most superficial description 
of Small Pouse impossible; it does, however, reflect a great deal about 
the attitudes of those dir^^ctly concerned with the operation of the 
program^ 

At the core of the program .\e d for Small House were two essen- 
tial concepts: flexible modular kiling and team planning^ Instead 
of four A5-rr.inute periods with fivt u.inutes between classes. Small House 
operated on ten 20-minute modules w.t^ no time between c^ ^sses„ (This 
was made possibly because of the physical proximity of the rooms to one 
another.) Of the 30 modules each week, approximatr ly 50% were required, 
with the other 50°; being spent in elect ives. (Sample schedules are at 
Appendix I and sample elective sheets at Appendix IT) Thus, in effect, 
students in Small House were only spending 507. as much time in tradi- 
tional classroom settings as their seventh grade counterparts who were 
not in Small House© 



1 

The report of Morris and Raain mentioned in the Foreword would probably 
include greater detail of event during the developmental stages. 
2 

In general, the electives were changed every week or two, although some 
were retained thraighout the year* 
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T^'O ri(.\iM( iHKlul.ir sc^^' chil iii<! wns V^c sfi*uclnr;i] cotiiix )n( n I t^al 
n:.-uK' it pi^Nsihlu ^\>r t^e Si:!n1l ]U)i:s<. to -icoc^;,.!)! i sonu of L^^ir 
y^)i\] s inci uf! i n;.: i iicn nsc d p/u*' nia 1 :v\(] co^.rnun 1 1 y i nvol vt :r;c nt , pi nmu fl 
stssions for st'.:ci.nt u cll^iacl: , and n broncJunno of fi-c ciirriculuni — 
it is '5;t:ii:jt:cd L^^l so::;o SO tliffr r.nt e lect ivos wc>r(> offi rrd ove r 
t^M- course- i)i ti-c ytnr. 'Hw Smn 1 1 Hcuse t( niii also cKvcloiK^d n nev; 
approiic^ tow:n-ds st:\ld.■n^ t valuation (ustd in parallel wit^ l^o Lra- 
dilional nK-t^-od sine- hoL^ slud. nts and parents in soitk' cases expressed 
a drsire for Iraditimal grades) and, in some classes, imp] emenl.r^d a 
different' p^ilosop^y of testing',, owe w^m c^- was essentially criLerion- 
re f t renccd«, 

Just as flexible modular sc^^edul i nci; was t^c skel-ton on i c^ 
Small 1^nls. v;as built, t^e "plnnnina team'* provided t^e >^eart and brains 
of t:^e sysLiiv,; t^. instructors used t^-e planniny^, sessions to discuss 
issues, develop tlecLivcs, ^andlc admi n i st )*a t i vc details, and assist 
eac>- ot^^er in prof- ssional r.rovt^. V-e teac^M?rs ti^niselves strrssed 
V-c contribution of t^e planning, team to Small House succosses^*^ 

students sele cted to Participate in Sn\n 1 1 House represent a 
quasi-random samplft of all t^os»^ studrnts v^^-ose parents indicated 
3 willin^'ness to ^avc t^c^ir c^ildrMi participate. Of t^-e more t^an 
300 Icttirs sent out (one to every incoming 7t^ grader), some 200 
rt'plies indicated a willingness to participate^ from tin\se 200, 
100 students were selected on a quasi -random basis, wi t^ special 
attempts being made to assure a racial, sexual, and be^avi or-probl rm 
ba 1 anc^ • 

In summary. Small House is identifiable admi n i st m t ivel v as a 
program w^ic^ utilizes tv.e flexible modular sc>^Gdul inj> concept and 
team planning,. T^e psychological environment and characteristics 
arc directly related to t^e goals of Sm^nll House and will be discussed 
i n a later srct i on^ 



B. HISTORICAL PERSPECTIVE AND REVIEW 

America's purported propensity for educational innovation and con- 
comitant concern for evaluation notwithstanding, t^ere is a paucity of 
systematic research on innovative educational pro.c^rams (Carbone , 1 962 ; 
Hillson et^al,I964; Westbury, 1970). Westbury (1970) reports t^^t over 
a three-year period ne^t one evaluation study appeared as a topic in any 
of eight major educational journals— despite the fact th^t evaluation 
ccnt^riues to be one of the most frequent subjects of papers*, This leads 
to the conclusion that evaluation is like the weather , i ^e*, , everyone 
can talk about it but no one does anything about it! 



The read-r is a;?ain referred to thr p.iper by Radin and Morris for 
elaboration; also, some of this will be addressed in Chapter V. 
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_ T^c clfni-L^. of rest arc'- is p.-,rt i cu 1 nr 1 y ncntc -,,\t^ rpspoct to t^c 
junior scKoolf;; except for so;rr cK-scr i pt i v(- sLncHes (Stewart & S^nrk 1971- 

M.rconn t-t, G.K, , 1069) nnd sporadic i nv. st i ,n t io ns of tl-nn t.ncMn. '''''''' 
(ZvtMbclson rt „1, 1965;Gnn,sky,1970; Boylcs, 1968; Fn.cnkH , 1 968) , tK- 
rrsonrc*^ focu.in, om ,M junior H sc'-ool i .s virtunlly nonrxistcnt! 

orcforc, it will br necessary to review r( st arc'- from n mimbt-r of 
cIiEfcrcnt areas and extrapolate from t'o sr- results to t'^r junior t^igt^ 

Most of f-e evaluation rcsenrc'^ and reports centering on innovation 
are directly re ated to t^-e "nongraded vs t-c graded" classroom. Bockrat^ 
(1958), InRram(1960) .nnd SSnpl sk i ( 1 961 ) found significant diffc-en-es 
in reading favoring f-r nongraded classes. nockraf^'s rescarc*- is'of 
special intrrest sinc^ H^r report covered a period of t^^ree years and 
included n^s of 5,000+ and K,000 + , Carbone (1961) reports tKi t a 195-^ 
comparison of 99 nongraded students wif- 123 controls s'^owed readinr ^ 
ac»^,cvcment and personality adjustr^nnts vere better for f^e nongraded 
students; r^oreover, St,.re(1972) sur.n,irizes ot'^er studies fror. Apploton, 
Wisconsin and Brllevuc, WasMngton, bot^^ of wM c^ support t»^e contention 
t at nongraded students demonstrate greater gains in reading. Buffe (1962) 
found tKat nonRraded students made greater gains in mat*^ and mental ^ealtK 
Hillson ot^(196M did a caerful study, controlling for many extraneous 
variables and found significant differences favoring t^e nongraded classes; 
a follow-up study, t^owever, indicated t^at t^ese differences disappeared 
over time (Jones, et_^, 1967). More recently, Engel and Cooper(1971) 
and Morris (1971) found si gni ficant differences favoring t^e non- 

graded structure. Not all of tt^e researc*^ results »^ave been in favor 
of t^e nongraded classroom. Jones et al (1967) found no differences, 
as did Hopkins (1965); Carbone (1962) found t*^at in all areas of ac»^ieve- 
ment, graded pupils vere , si gn i fi cant ly better thin nongraded w»^ere t^e 
original scores vere adjusted for intelligence; t^^ere vere differences 
in tt-e ways t».e two groups of stdj ents described tt^eir teact^ers, wi tt^ 
tt>e nongraded students using more favorable terms to describe t>^eir 
teacf'ers » 

T^cre are several critical inter-related issues v^^ic^ must be 
kept in mind wi t'- respect to t^e cited researc*- : 

a. A.s Good lad (1968) ^^as pointed out, t^c distinction between 
t*^e nongraded and "traditional" classroom is often a fuzzy, 
if not tD tally arbitrary, one. Label 1 ing can ^ide t^e fact 
f-at some "traditional" classes '-ave more in common wif- 
oti-..r classes d. ignated as nongraded f-an ot'-er classes 
designated as graded. Engel and Cooper's (1971) recom- 
mendations constitute oniy a partial solution. T>-is criti- 
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cism CiHi Ik broachncd t incluclG cl i 5: t i net i ons nnionp, educn- 
t:iotinl t-rt_cil-i.ic'nts (or curricula) in p^cncrcil; n ssort: 
di f r-rc-ncrf. iLJiLLflLl (^^V Ccillinp, t^ein different names) 
vir> little or no basis for doino soc 

b. M. t^odolo^'.ical and de^ip.n flaws c»-aracteri ze most of t^e 

rc>5u>arc^» Almost all of t^c research is restricted by a model 
v'^ic'^ »^as pone little beyond t^o concept of paper and pencil 
tests; l^e failure to incorporate systematic observations as 
an intr;pral part of a compre^^e ns i ve model ^as been t^e most 
^',lnring defect of educational evaluation research, 

Cp In many casfs, t^o research covers a brief period ( 1 year); 
t^ose cases \<!^\c^- ^ave covered a longer period (l^opk i ns , 1 965 ; 
Jones, et al, 1967) ^ave indicated no differencesc 

Like t^e research- on t^e nongraded classroom, t^e research on team 
teaching produces different results. Zvplbelson et al (1965) found no 
c^^anges in achievement, but improved attl tides on t^ose under t^p team 
teaching meu^od. Gamsky's (1970) findings corroborate t^ese results o 
Rhodes (1971), on t^e ot^er hand,chows t^at t^e average reading gain 
was worse for team teaching, as XN;ere changes in pupil attitudesi ' T^v 
criticisms directed at research on t^e nongraded classroom are appli 
cable to t^at focusing on team teaching. 



C. RESEARCH EFFORT 

Commencing in September of 1971, t^e author began meeting regularly 
wit»^ tv^e trackers and spending time observing and taking notes in t^e 
classrooms. Observation periods were random or nonsystemat ic two or 
tv^rce times per week and continued t^roug^ June of 1972« T>>ese notes 
form t^e basis of Chapter V and it x,7as from t^ese notes t^^at t^e 
questionnaire (Chapter III) and t^e proposed observation schemes were 
derived. All of t^is, however, was directly related to t^e goals 
w^ic^ tVe teachers ^ad set for themselves (see page 1). We did not 
succeed in addressing ourselves to all of t^^e goals set fort^ and 
developed by t^e teachers, at least on a thoroughly systematic basis; 
we emphasized those that were essentially observable and those that 
had been given higLer priority by the Small Pouse team, (T^e goals, 
as they were statod in their original formulation, are at Appendix III«) 
These goals could be rephrased to read: 

a* Do Small House students s^ow gains in performance on standardized 
reading and arithmetic achievement tests comparable to those of 
students in other classes? 



b« Do Small "ousp students s^ow a more positive attitude towards 
school as compared to students in other classes? 



1. 



c. Arc. t'-or; cKiract . r i st i c diffcrcncos in teacher and student hc'-nvior 
iH.trwc-on Small n^usc^ and olSt olass-s? Tf v.-Kat: arc t:i>eyV How 

nr.- t fv related Lo V^v f-.tvals c.f t:*^f i-uman, nc.ss report? 

A severe li.ntat:ion of t'-is study war. t'-nt all of t'-e control classes 
were matK nut ics classes, Tw i s nvidc it impossible to compare tt-r. obser-' 
vations of ot'-er Unified classes and .Science classes wi t>- t>os< of Small 

MOUSC • 
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CHAPTER TI 



PSYCIIOMETR T C EVALUAT T ON 



A. CALIFORNIA READING TEST 



Among t^e goals t^cit t^c teachers ^ad set for themselves were t^ose 
related to academic achievement, particularly improved reading skills. 
The Small Vous(i team vas particularly concerned t^a t students in Small 
House show no decrement in performance in the basic skills as compared 
to the gains reflected by a control group, despite the fact that the 
former were spending less time in traditional classes. The California 
Reading Test (1963 edition. Form W) was administered to all the students 
in Small House and to a small sample of those in the control groups-'o In 
order to fit the test into the A5-minul:c prrriod allowed, some modifications 
of the normal testing procedure were required: 

a. Only the Vocabulary Test and Interpretation of Materials subtests 
were administered. 

b. Students were only given 30 minutes to respond to items in Section 
G of Test 2, Interpretation of Materialso 

c. In scoring these tests, no correction was made for guessing© In 
addition, each correct response in Section G was given a weight of 
two (2) poi'ntL to overcome the preponderance of vocabulary items 
(60 vs 45). 

Table I presents the results of thp groups' performance on both the 
pre- and posttes^. It appears that there are no significant differences 
in the net gains made by either group. Because the control group was 
expected to be more homogeneous at the beginning of the year, it was 
decided to use Analysis of Covariance to compare the two ^>,roups. In order 
to achieve parity in group size, eac'- student from Small House for whom- 
pre and posttest reading; scores was available was randomly assigned to one 
of fo» - groups; thus, four separate analyses of covariance were performed, 
comparing each of thr experimental groups with the control groupc In effect, 
we have four tests of the hypothesis of no difference. Tables II - V 
present the results of these comparisons. 

Nonr of the F-ratios obtained is significant; nevertheless, there is 
one interesting difference that emerges from this analysiSo The variance 
of the control ,-;roup on the posttest was almost double that of the control 
group on the pretest. But in each of the four experimental groups, the 
variance on the post test was less than than the variance on the pretest! 



It w.'iS possible to only use a small sample due ♦'o the fact that these 
students were also taking othrr trsts and it was n^-^cessary to minimize 
demands on class time* ^ ^ 



TABLE I 

PRETEST AND POSTIEST Ml£A\'S (RAW SCORES) ON CALIFORNIA READING TEST 



GROUP 


N 


PRETEST MEAN 


POSTTEST MEAN 


NET GAIN 


SH'XLL HOUSE 


85 


77.7 


90.2 


12.5 


CONTROLS 


20 


79.2 


92.6 


13. A 



TABLE II 

ANALYSIS OF VARIANCE FOR FIN/\L READING SCORES BY COVARIATF ADJUSTMENT OF 

INITIAL READING SCORES 



SOURCE 


df 


SLIM OF SQUARES 


MFAN SQUARE 


F-RATIO 


BETWEEN 


1 


60 


60 




WITHIN 


37 


4307 


117 


,57 


TOTAL 


38 


4367 
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ANALYSIS OF VARIANCE FOR I^INAL READING SCORFS l.iY COVARIATF ADJUSTMFNT OF 

INITIAL R FADING SCORES 
(SM/\LL HOUSE SUBGROUP 2) 



soin^CE 


df 


SL'M OF SQUARIIS 


Ml: AN SQLIARI' 


F-RATIO 


BLTWEliN 


1 


281 


281 




WJTIITN 


37 


3995 


106 


2.76 


TOTAL 


38 


4276 







TABLE IV 

ANALYSIS OF VARIANCE FOR FINAL READING SCORES BY COVARIATE ADJlJST^^ENT OF 

INITIAL READING SCORES 
(SMALL HOUSE SUBGROUP 3) 



SOURCE 


df 


SUM OF SQUARES 


MEAN SQUARE 


F-RATIO 


BETWEEN 


1 


27 


27 




WITHIN 


37 


2616 


72 


.37 


TOTAL 


38 


2643 







TABLE V 

ANALYSIS OF VARIANCE FOR FINAL READING SCORES BY COVARIATE ADJUSTMENT OF 

INITIAL READING SCORES 
(SMALL HOUSE SUBGROUP 4) 



SOURCE 


df 


SUM OF SQUARES 


MEAN SQU/\RE 


F-RATIO 


BETWEEN 


1 


14 


U 




WITHIN 


37 



3320 


89 


.15 


TOTAL 


38 


3334 
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rr arc several viable explnnations Tor t^is p'-cnonirnon : 

fl. Tt Is r.iniply a r. Rr., s^:lon cffr-ct w'-ic*- is not rvidfnt in t^c conLrol 
group since t->c-y arc more *'Dnios:;r.nf.f)us and, on t^c avcrapc, w€Te 
atnonn t»>- more i «H y motivated and more able student s — accord i nj; 
to t»"^ir placement, 

b. T'-at, in fact, t'-e brighter students gained less — in an absolute 
sense — t>'an f-ose w^o scored lower on t'-p pretest. T*- 1 s is a 
reasonable assertion (within limitc.) becaus- of t^-c intensive 
programs tt^at Small House '-ad for students wt'ose basic skills 
were in need of remedy. 

It is critical t'-at certain limitations be kept in mind in reviewing 
t'^e results of t^-is portion of tt^e research: 

a. Tl-e number of students in tt^e control group was smaller (n-20) 
and miic*' more homogeneous than t^c students in Small House." 
This suggests that the ^!;roups were not comparable at thf outset, 

b. Testing conditions, particularly in Small House, w«;re somrw^-at 
less than ideal, and not standard; in one case, due to interrup- 
tions, it was necessary to rcadmlnister thf test, 

c. Although tests were de-emphnsi zed for all of the students taking 
them, students in Small House, especially on tv-e pretest, expressed 
the belief that their performance on the teslH would affect the- 
future of Small House. It would be impossible to gauge t^r precise 
effect, if any, that this migi-t hnve had on their performance. 

Given th,. limitations of t'-is portion of the study, the results must 
be considered tentative, at best, Tn general, t^cy are consistent with 
the trends of t^e research on f-e nongraded elementary classroom. They 
shore the common prr.blem of rcsearc*- on t>-e nongraded classroom in that 
a one-year period is 1 lumf f 1 clent time to adequately asnesr. t'-e impact 
of any broad-based social program, Tertalnly, pome of t'-c findings arc 
provocative; if, for example, a follow-up study also discovered t'-af 
a control group tended to show greater variance on t^-e posttest f-an 
on a pretest w' l le Small H^use students did tv-e opposite, we could be 
far more confident of these results. It s'-ould also be kept In mind 
thflt the Small House teachers, w'-lle wishing to open up t^c educational 
experience and provide diversity for their students, specifically 
implemented programs designed at students wU> problems In the b/islo 
skills. Prom an evaluation cr)ntcxt: and In terms of understand 1 nj/ 
precisely what f^e treatment consisted of, t'-ls is a very cogent 
vo r 1 a b 1 c . ' 



Other cx/impleft will be |)rovlded and cll;,eu;uied In Chapter V. 

l.'i 



B. CALIFORNIA /.rtrriMETTC TLST 



In addition i '^c\r concern wit^ rending skills, t^e Small House 
team wns concerned vit^ improvement in bnsic mnt^^cmntics skills as 
compared to students in ot^^er classes. Br sides t^^e fact t^at t^e 
students would be spending; only nbout 601 as muc^ time in mat*^ classes 
as t*-eir 7t»^ grade counterparts, t^c Small House mat^ program was 
constructed armnid individualized programmed instruction kits. T^us, 
t»^e basic nuxic f instruction was radically different in conception 
from t^e "traditional" 7t^ grade mathematics class. In addition, 
t^oug^i, sprcial remedial instruction was offered for t»^ose w^o needed 
It. 

T^c California fy^}^met\c Test (19^3 edition. Form W) was administered 
to all of ty^c studerit^s in Small House and to all students participating 
in t»-e control groups. In order to fit t*>e test into t^e A5-minute 
period of the normal classroom schedule, some modi f lead ons in admin- 
istration procedures were required, 

a. Only t^e computational portion of t^e test was administered^ 

b. The time limit for cac^ section (Addition, Subtraction, Multi- 
plication, and Division) was 10 minutes instead of the 10,10, 
12, and 15 used in the standardization. This hos a bearing 
only If we wish to examine the mean grade level placement or 
compare these students with the national norms, neither of 
which is a hjgi, priority goal of this research. 

<:» No correction was mide for guessing. 

Attention was restricted to computational ?i<llls because of time 
limitations and the fact t^nt the teachers cooperating in the research 
indicated th^t these were the skills emphasized In 7th grade. As with 
the reading test, students expressed their belief th.n their perfor- 
mance would hflve significant bearing on the future of Small House; 
other th^n thnt, testing conditions could be considered standard for 
all groups. 

The control group consisted of three different first-hour classes, 
taught by different instnictors who also volunteered for observations and 
attitude testing of their students. One of the three control groups was 
designated a 372 (low level) class, while the others were designated 
37A (average ♦). For th hi reason, the analysis includes a breakdown 
by class, in addition to combining the thrrr classes for comparison 
with Small House.. Table VI prrsetits tHo pretest and posttest means 
for the grai ps Involved. 
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TABLE VI 

DETEST AND I'OSTTEST MKANS (KAW SCORES) ON CALIFORNIA ARITHMETIC TEST 



GROUP 


N 


PRETEST 
MEAN S.D. 


POSTTEST 
MEAN S.D. 


NET GAIN 


CUNrROL A 


1 9 


25. A 


8.25 


37.9 


U.l 


12.5 


CONTROL B 


26 


A2.6 


8,35 


55.8 


10.1 


13,2 


CONTROL C 


2A 


39.3 


10. 3A 


53.0 


9. A 


13.7 


COMBINED 
CONTROL GRPS 


69 


37.1 


llc76 


50.1 


13.78 


13.0 


SMALL HOUSE 


83 


37.1 


13«72 


50.1 


12.53 


13,0 



TABLE VII 

PRETEST AMJ POSTTEST MliANS (RAW SCORES) ON CALIFORNIA ARITHMETIC TEST 

— LOW ABILITY STUDENTS — 



GROUP 


N 


PRETEST 
MEAN S.D. 


POSTTEST 
MEAN S.D. 


NET GAIN 


CONTROL A 


19 


25. A 


8.25 


37.9 


lA.l 


12.5 


SMALL HOUSE 
SUBGROUP 


19 


26. S 


9.01 


38. A 


9. A 


11,9 
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T^cro nrc scvrrnl notcworti^y pc cul inri tics of t^is tnblc c. T^o 
firsts of course, is t»-nt w^cn t^r> control clnsses .-rrc. combincc) t^e 
protr-^t nnci post test means nrc identicnl to t>-os(? of Snunll ^^:)iise 
students^ T^e second is t^nt roR.nrd 1 c S5: of v;^ i clnss was involved, 
all wit^' diffprent instructors, t^e mocin r/iin for cnc^' class was 
approxiniatel y t^c same, irrespective of cla ss mean on t>-e pretest ! 
Yet another is t^e tendency for t^^e postteyt Vcirianc^- to be greater 
t^an t^r pretest variance for t^e control group w^ile t^e converse 
is true for t^e StnaH House students* i s latter is particularly 
evident in Control Group A, t^e low ability classc It was decided, 
therefore, to follow t^^^ recommendations of Campbell and Stanley (1963, 
p 185) — albe it for t^e wrong reasons— and compare t^e performance 
of t^-is class wit>- a comparable sample drawn from Small Housec 

Table VTT presents t^e results of t^is comparison. T^e subgroup 
drawn from Smn II House tends not^ to s^ow t^e large increase in variance 
reflected by t>^e control group. An additional analysis of t^-e perfor- 
mance of ty-c low ability students in t>^e ot^^er control classes s^ows 
a tendency for t>^em to fall furti^er be»-ind t^-e mean over timeo T^ese 
findin^»,s were substantiated to some degree w^-en t^e trac^-ers identified 
t^osc students w^ose scores s^^owed t^e least improvem(?nt as U^ose w»-o 
»-ad,in t»-e eyes of t»-e teachers, learned t^e leasts 

Like tVie previous test results (California Reading T^st), t»-ese 
data s»-ould be read witv» a jaundiced eye. Even t^oug^ t^ey are based 
on a ,slig»-tly larger N, t»-ere are still some questions as to t»-e 
essential comparability of tt^e groups on variables ot^-er t»-an mat»-« 
(N.B. All of t»-e students in Small House ^ad parents w»-o volunteere d 
to participate.) T»-ere may »-ave been motivational diffr^rences between 
tHe two groups., T»-erefore, promising t»-oug»- t»-ey may seem, t»-esc 
results s»-ould be consid^ rf>d tentative at best« 



C. SCHOOL SENTIMENT INDEX 



Consistent wit»^ t^e humaneness report, one of t^e primary goals of 
Small House was to effect changes in t^e attitudes of students towards 
sch ool — to "turn students on" to sc^ooK One of t»-e ways selected to 
measure t^is c»-ange was t^e School Sentiment Index (SSI), a criterion- 
referenced measure developed at t^e Instructional Objectives Exc^^anT^e at 
UCLA., Additionally, ve set out to develop an observation sc^^edule • • i c^- 
would supplement t^e paper and pencil measure; »-ope fu 1 1 y , t^ i f; would 
enable us to quas i -va 1 i da te t>-e SSI (i.e., students wi t^ low scores would 
demonstrate behaviors distinct from t^ose wit^ ^ir,^" scores). Moreover, 
it was also v»oped t^at we could use some of t^e test items for inclusion 
in t^e observation sc»-edule. Technical difficulties delayed t^e completion 
of t^e observation schedule to t^e extent t^at it was only used nonsystcmat i ca 11 y 
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find will be included in C»-fiptcr V. 



One final gonl of t»-c use of .tH. SST was to identify some smaller number 
of Items „?.)c^ would be more amenable to classroom administration, be 
sensitive to c»^ange, and yet still be reliable 

1. P rocedure . T^e SSI was administered to all students in Small House 
in Septe.nber and t^e control group in October. 5 T^^e posttest was administered 
to bot^ groups in Mny,1972. T^e following instructions were read to t^e 
students; 



We. are interested in w^at 7t'' graders tMnk and feel about school, 
t»^eir teac'-ers, and t'^eir classes. Therefore, we would like you to 
read eac'- statement carefully and indicate w'-et'-er you agree or 
disagree wif- it. If you agree, circle yes on t^e answer s^eet; 
If you do not agree circle no. Please make sure t^at you are 
circling t^e correct response-f-e number next to t'^e answer you 

TUTc^'ic^Un^-"'^""^'^ "^'"'^ '''^ ""'^^^^ "^'<t to t^e statement. 

TMIS IS NOr A TEST, THERE ARE NO RIGHT OR WRONG ANSWERS. N^ne of 
t*^e teac»^ers or t»-e principal will see w»-at you answer! 

T^e major difficulty wl t^ f-e test was interpreting t'^e meaning of f^e 
word teachers. For t^^e Small Vouse students, it referred only to t^-eir 
clustf-r teaC-ers and did not include electives. For t'^e students in t'^e 
control groups, it referred only to instructors in t'^e basic subjects. 
Ma t»^, Science, Social Studies and English. Nevert'^e le ss , students in t»-e 
contro groups made it eminently clear tKnt t'^ey considered it unreasonable 
to reply to t»^e items dealing wif^ teachers in any generic sense. T^is 
was not t»-e case wit^ Small House. 

2, gesLilts . Table VIII presents t^e pretest and posttest means on t^e 

f u '^'^^'^ '"^""^'^ ^'^^ proposition t»^at Small House 

students s'-ow a significant gain in attitude between pretest and posttest 
wM e tr.e control group does not. Tablr IX summarizes t^e four basic com- 
par sons (t-tests) made. In order to MgMig».t tMs comparison, it was 
decided to compare students in bot^ groups W^ose scores changed significantly, 
+ n r !;n°^r' '^'^^"'^ categories were arbitrarily defined as 

+11 to +30. +5 to +10. -A to +A, -5 to -10, and -11 to -30. Table X presents 
t^e results of tMs breakdown; since tH're arc different numbers of students 
in t^e two groups, entries will be expressed as percentages^ 

Significantly more students in Small House made gains ot 11 points or 
more on t»^eir tests relating to attitudes towards sc'^ool, as measured by t^c 

fn7°nf !l ?T ''^^^ "^'^ reveals, however, t'^at approximately 

10/. of t^e students ln_c: g rou^ showed poorer attitudes towards school at 
t^e end of t'^e year t^nn t^ey ^ad at t^e beginning. In some cases t^ese 
c»^anges were substantiated by observations w^^ic'- were conducted t^roug'^out 

5 

T'^e reason for t^^e difference in administration tines was f-at t^e Small 
>ouse students were going on a camping trip before October and it was considered 
necessary to minimize euphoric effects based on a single experience; however, 
the control groups could 'not be tested at f-e same time because of t'-e need for 
Pv',rentfll permission, ^ 
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TABLE VI T I. 

PRETEST AND POSTTEST MEANS (RAW SCORE) ON SD'OOL SENTIMENT INDEX 



GROUP 


N 


PRETEST 
MEAN S.D. 


POSTTEST 
MEAN S«D. 


NET DIFFERENCE 


SMALL I'OUSE 


83 


71.1 




76.7 


12.6 


5,6 


CONTROL GRP 


69 


69„3 


13.2 


70.7 


11.7 


1.3 



TABLE IX 



COMPARISON 


M2 - Mj 


t 




Small House-pretest vs Control s-prrtcst 


U8 


• 82 


not significa 


Small Housc-posttest vs Control s-post;tcst 


6.0 




significant - ,01 


'small lIousG-posttcst vs Small Housc-prcUest 


5.6 


2.67 


signi f icant - .01 


Controls - possttest vs Controls - protest 


U3 


.71 


not signi f icant 



TABLE X 

ALGEBRAIC DIFFERENCES BETWEEN POSTTEST AND PRETEST SCORES ON SCHOOL SENTIMENT 

INDEX 



NET CHANGE 


SMALL HOUSE 


CONTROL GROUP 


+11 to +30 


27.57. 


8. 37. 


+5 to +10 


1 9. 87. 


20 17, 


-A to + A * 


A 2. 97. 


60.97. 


-5 to -10 


3.57. 


6.57. 


-11 1:0 -30 


5.57. 


3.97. 



Thr t-tcst for corrol/itpcl means wns iiscd for i s comparison 
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t-'v. scH.c.I yc.;,r on n no„sy.stcn,nt i c bnsir, tH^ nntiirc of t^e d-nnpc l.cinc, 
conson.-mt w,t^ ,>.c dn-cction of tSc> cKnn.c of L-^eir nttit.de test score.' 

One ..Klditionnl nnnlysis wnn concluctc-tJ in ,m effort to identify t^e dimen- 

ttr^inlv , f-T'^'" ""'^ «roup.s~to identify 

ind, vuK.nl. lU,.^ on wri t'-c-re wns a sij.nificnnt difference between t'e 
two groups on tK^ po.sttc-.st t^ou.> no significnnt differences existed on t^e 
pretests Inble.Xl presents t'^c results of Hms nnnlynis; t'^ese results are 

TtlnTt\T ''f ^'^'■^^'^^'^^^■•^ '-'"^"n. t».e control groups on son,e 

terns wns ns lar.^e-or lnr,,er-tKin t'-e differences between Smnll House and 
t c control group t.ken .s n w'-ole^ Table XI presents t^^ose i ter.s t»^at were 

"f'u.j;::r: :s?n,r,aVt°:e:dr'^ "^^-^ - ^--^ 

Tt^e.se results nre probably t'-e most ir^portant findings since tW^y do 
identify dir.ensions wH.re Small House --as s'-own consistent significart difference. 
wMc»^ nrr related to t^e goals t'-at t^e Small House team set for t^^emselves 
and also to t'-e broader issu. of •"-umaneness, " T^e one item wH c'^ runs counter 

ow rds '"tT'"" f'^^^^"'"'^---'' ^T'-c otwer kids in my class are not friendly 

cowards me.). Tl is also important to note tKit t'-ere were vast differences 
among t«-r control classes along f-o same items; tMs simply veriTics observation^ of 
T^^rTir" '-'^'"^■^ ^^^-^^^^ combinations o^ in.cructors and t^e 

suulents. combinations of instructors will be viewed differently by different 

Tmlnv'--Tr^r^f^' ''''•^"^ •■'^ortcomings of t^e School Sentiment 

Index Ks Knt it uses generalized refrrrents (our teachers are . . .; our classes 
Z\r-\n i.^'Tl f^cq^cntly expressed t'-eir feelings tKit it was- not 

ffllr to ask t^em to lump teachers and/or classes together and judge t^em, 
tKit the only reasonable responses tKit t'^ey could make would be to assess a 
part cular teacSer or a particular class. TMs is a legitimate complaint', but 
1 w<.s necessary to work wltMn the constraints of the system wMc" included 
eliminating references to individual instructors. 

"v. .""0^"' by ■students concerning the SSI was the simple 

ys or no answers required. T'^e student s fe 1 1 th.t i t wou Id be more appro- 
priate to h„v,. n-ore categories of responses, "most of the time", "about h he 

ime , Hardly ever", "never." Moreover, t'^e students felt that the tests were ' 
too long (a problem we h.^d anticipated) and , recommended . 

One of tNe major problems with th)s study is th.^t the control groups 
represented a sampling of instn.ctors which u^^y ^,ve been biased by unknown 
factors; t is not safe to nssum,e tKit at least one representative of nil 
po.ssible instructor combinations was pnsent. Interpreting th,. results t'-en 
becomes a matter of believing tS.t t'-ose students in t'-e control group represented 
n random sample of Slauson 7th graders and t'-at t^eir instructors for thoir 
,ns e courses represented random sample of possible combinations. Of-erwise. 
it is difficult to ascertain w'-at it is t'-at Small House is better thnn. 

i!) 
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TABU' XT 

PlvRCKiNTACli: 01' STl^DLNTS RESPONDING AFFTHMATJ VlLY TO ITEMS ON THE SCHOOL SENTIMENT 

INDEX 

ITEM SMALL HOUSE CONTROL GROUP 

I usucilly look frjrwnrd to corr.iti^ to 

scV^ool 76 58 

T^-is *;c1iool is like ri jciil 12 30 

T^^cre cHTG many different activities from 

w^ic^ I can choose w^at I would like to do 96 61 
My teachers try to make school interesting 96 72 
My teac>'crs treat mc fairly 92 66 

My teachers try to make sure I understand 

a^at t^cy x^;ant mc to do 70 51 

I ^et tired of ^carin^ my teachers talk all 

t^e time 10 5A 

T^'e ot^cr kids in my class are not friendly.^ 

towards mc 29 13 

Our teac^M-rs usually listen to our ideas 90 51 

We usually ^avc fun in our classes as well 

as learn 88 66 

Most teac*-ers ^avc no sense of ^umor 2A A9 



TABLE XTI^ 

PERCENTAGES OF STUDENTS IN EACH OF THE CONTROL GROUPS RESPONDING AFFIRMATIVELY TO 

ITEMS ON THE SSI^ 



ITKM 


CONTROL 


CONTKOL 


COMTR OL 




A 


B 


C 


Tl^ls sc»-ool is like .n jnil . 


45 


18 


28 


My tcncl-it. r s try to mnke sc'^ool i ntr rc st i n^'. 


82 


73 


62 


My tc'rnc''cr.s do not cnre nbout t'Mir students 


31 


13 


26 


My ttnc^crr, trc.nt me f.nirly 


68 


72 


8/« 


Most Icnc'-'crs try to >^cl]i you 


100 


80 


100 


r)C*-ool \n n'.or.tly a w.'ir.tc of tixc 


25 


00 


12 



All diff t'TcnCfs r» i ^'u i f 1 ca nt /it #01 Irvcl, 
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CHAPTER III 



THE 2A-H01IR QUESTIONNAIRE 



Rc-ference was mnck> carlifr to the abortive attempt to develop an o'.servat •• on 
sc»-eclulc to ho usfd 5n tandom v^it^ t^^e SSI to a.sses.s attitudes towards sc'-ool; 
tt^is observation sc»-edul- was to focus on t^o be'-aviors of individual students 
to dfterminc- if t^cy coincided vif^ certain criteria w'-ic*- t^e instructors and 
rcscarc'- staff ^ad agroed would ref lectcertain kinds of attitudes 
Wt^ilr tt^is was not fully utilized, it did load to using observations to con- 
struct a questionnaire for students to respond to wif- respect to specific 
events t^at "^ad occurred wiL»^in a 24-t^our period. T^us, t^e students provided 
observations of w^-at ^ad ^-appcned to t^-etn. T»^is was not intmded to be another 
paper and pencil test of attitude alt^oug^ it is certainly a useful adjunct 
to tv-e Sc'-ool Sontitncnt Index. But essentially it was ^oped t^at t^is method 
would avoid t^ generality of t^e SSI. 

1. Administration, T^p 24-^our questionnaire was administered twice to 
each participating class-oner in t^e first week of April and again in t»^e 
last week of April. It was always administered prior to t^e beginning of 
each class (eac»^ student's first class in the morning) so as to minimize 
differences and possible misinterpretation. T^e following instructions were 
read to t^e students: 

"We are interested in some of t^e things t^at happen to 7th graders during 
an average day. Listed below are a 'number of things t^^at could possibly 
have happened to you or t'^at you could have done in the past 24 hours 
(since you came to school yepterday morning). Please indicate whether 
these things did or didn't hgppen by simply circling yes or no behind each 
statement. For example: 



I went to see my counselor yes 



no 



If you saw your counselor anytime yesterday or this morning, you would 
circle X£s, otherwise you would circle no. Remember, all of t^e items 
refer to things t^at happened between yesterday morning and this morning. 

2. Results. Like the results of t^c School Sentiment Index, t^e results 
of tv-is brief questionnaire are botv- provocative and valuable. To a large 
extent, t^ey substantiate f-e results of t^e SSI— even tt^ougv- at least 6' 
montKs intervened between t^-e two administrations. As witt^ f>e Sc^-ool Sentiment 
Index, t^e Small House students reflected more negative peer interactions 
(itesm M. and //9) than did students in t^r- control group. Seven of t^e 
items yielded significant diffcrrnces among t^e groups on bott^ admi n i stro t i ons. 
Table XIII summarizes t'-e rcr.ults of tMs questionnaire. Iron i ca 1 ly, Sma 1 1 
Housr students also respondfd significantly better to item 8 ( A student did 
something nice for me) indicating an increase in positive social interaction. 
The ol^er dimrnsiois of difference appeared to be one of boredom (#2,//5, and 
#10) and positive tcac';cr activities (//i and iHl). 
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TABLE xni 



PERCENTAGE OF STUDENTS IN SMALL HOUSE AMD CONTROL GROUP RESPONDING AFFIRMATIVELY 

ITEMS ON THE 24-HOUR INVENTORY 



1. 



ITEM 



A teacher said something, nice to me-- 
something that made me feel goode 



WEEK 1 
SMALL CONTOOL 



WEEK 2 



HOUSE 
57 



2^ I got very interested in something that a 56 
teacher was talking about and wanted to abk 
(or did ask) a question about it. 

3, I volunteered information based on my 32 
personal experience or something I had read. 

4, A student said something unkind to me or hi 
somethi ng that made me feel bad, 

5, I was very b';)red during a class and spent 48 
most of my time looking out a window, day- 
dreaming, or talking to someone. 

6, Another student said something nice to me 64 
or something that made me feel good. 

7, A teacher hollered at me or got mad at me 16 
for no good reason and wouldn't even listen 

to my explanation. 

8, A student did something nice for me. 60 

9, Another student or group of students made 28 
fun of me or picked on me. 

10. I couldn't answer when I was called on 13 
because I wasn't paying attention or I 
didn't do some reading I was supposed to do. 

11. A teacher volunteered to hblp me with a 45 
problem I was having with my schoolwork. 

12. A tpncher said something unkind to me or 16 
something that made me feel bad. 



GROUP 
38* 

40* 

20 

26* 

73* 

66 
20 

43* 
11* 

13 

23* 
14 



SMALL 
HOUSE 

51 
57 

36 
43 
56 

61 
23 

62 
27 

10 

46 
19 



CONTROL 
GROUP 

30* 
35* 

IB 
19* 

76* 

63 
17 

44* 
13* 

24* 

28* 
13 



* - Statistically significant at .01 level, 
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3. Discu ssi on. Tn nddition to supportinf-.-.nt ] Cci.sC in pnrt-t'-e 
results of t^c. School Sentiment Index, t^c results of t^is quest i onm, i re 
<nre interest, nr, in t^c^ir own rij-t-t. For cxnu,ple, approximately 75% of 

rt ' "1 ^nv"'^' r ^"^'"fi course of a 

scKool day (30Z ,n Su,n 1 1 House); by contrast, we see tK.t only 17% of 
the students ndmit to being interested in anytMng tKnt was going on to 

' ""''"'"^^ '° ^""^ ^ question about i t (55% in Small 

House;« l^,s si'ould coi.c as no great surprise to persons familiar wi t»^ 
t»^e traditional fare offered by junior M sci^ools„ Nevertheless, it is 
thougt^t-provoking statistic wit*, respect to t^e breadt*. of t^^e problem. 
(Wl^at portion of t^^e day or ^ow often t»^e students are bored is an 
unanswered question) Similar comparisons can be made wit»^ ot^^er items 
— #1 vs #12 for example— to »^igMight ott^er points of interest. 

As wit»^ ty^e Sc'-ool Sentiment Index, t»^ere were differences among t»^e 
contro groups on some of tt^e items (if 2,5, and 7) w^mC^ exceded those t^, 
ex sted between Small House and t^e control groups taken as a wkile. 
This again illustrates t^e importance of t^e combination of teachers t^at 
kids have as well as t^c weakno.ses of using items with nonspecific 
referronts. 



The value of the 2/.-hour questionnaire is that it is highly specific 
and the items themselves rould be validated through observations" It is' 
short (requires only 10 minutes to administer) and could be administered 
several times throughout the course of a year. 
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CKAPTER TV 



SYSTEMATIC OBSERVATIONS 



1. IntrnducMon. Throughout the course of the school year, classes 
in Smnll House and t»^e control groups were observed in an attempt to develop 
classifications of characteristic teacher and student be»^aviors t^^at were 
hot^ compret^ensive and precise. At first, t^ese observations were non- 
systematic in nature; only later xoere t^ey formalized into an observation 
schedule. This schedule was finalized and tested t>^roug^ t^e tmntt^ of April. 

Beginning in May, all of t^e classes participating in t^^e study were 
observed regularly by two observers on f-ree separate occasions for a total 
of six 45-mimite observation periods per teac>^er. Each observation period 
included two lO-.ninute blocks of data collection; for t^e control groups 
t^is meant one block at t^e start of t^e class and t^e second block starting 
at the middle of t^e class. For t^e Small House classes, t»^is meant two 
consecutive modules of observing t^e same teac»-er. 

Eacf' lO-tninute block consisted of two parts. T^e first part was a 
coding of teact^er-student interactions (coding t^e be»^avior on every 15t^ 
second-20 codings per 5 minutes); the second portion included two scans of 
the classroom, noting w^at each student was cbing. Studeit -teact^er inter- 
actions were coded on the basis of v,ho initiated t»^em and wt^at the content 
of tKe Interaction was. Student be»^aviors were classified as traditional 
learning, nontrad i t i onal learning, socizlizing, observing, or daydreaming/ 
wU^drawal. (For a complete description of t^e Coding Sc»^eme, see Appendix 

One of the serious limitations of t>-is part of t^e study is t»-at all 
of the participating classes in the control group were wath classes^ and 
thav. comparisions t^^erefore are not directly possible between Small House 
and anot»^er total treatment combination, or oven between Small House Unified 
classes and traditional Unified classes. It is for t>^is reason t>^at 
t'-e original comparisnns are simply between t^e Small House math class^ 
and t'^e of^or t'^ree math classes. 



2. Procedure ,, ^s the number of naturalistic observations we >^ad made 
increased over time, we attempted to break t'-etn down into well-defined, 
mutually exclusive categories and f-en use f-ese to guide observations. 
The major stumbling block confronted was a band-wi dt'-'-f idel ity dilemma . 
In order to aC^ieve consistently ^ig^ agreement, we »^ad to widen the parameters 
of our categories. T'-e system selected was considered t»-e best alternative 
available from t^e perspective of clarity, distinction, and reliability. 
Subtler discriminations and specificity were temporarily sacrificed due 
particularly to time constraints. 
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Several classes w^jicl:^ initially agreed to participate in the study 
g dropped out, leaving only ti-c mat.'- classes. 

Only required matt- classes in Small House wore observed, not electlvGS, 
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Startinf. in April, t«^c txco mnin observers bognn observing classes in 
order to e.tab isv fundnmental reliability. Consistent witK t^e ^ccLnen- 

HbiHtv"' TV- ;^^^^)'^'-=^^'--y 0.967), and McCaw ot " 1 J) 
rcliabn.ty wa.s conceptualized as having f^ree distinct co.nponciTTT^r facets 

a. Tnter-observcr agreen,ent. Two observers in t>^e same class at t^e 
same t-ime will make essentially t^e same statements about t^o 
process, 

b. Tntra-obseryer agreement. One observer in t^e same class at two 
different times will not comprise a source of error, i.e., will 
be consistent in l^is use of categories. 

c. Inter-observer reliability. Two observers in t»^e same class at 
ai terent t imes. 

Medley and Mitzel (1963) summarize tt^e distinction t^^us: 

"u : '/^^e coefficient of observer agreement tells us sometJ^ing 
about tv-e objectivity of an observational tecKnique; tf- e coefficient 
of stability tells us sotnetMng about tM^ consistency of t».e 
bel^avior from time to timelO But only t^^e reliability coefficient 
tells us '-ow accurate our measurements are." (p 254) 

T»-e coefficient of inter-observer agreement (C, ) was calculated 

(Ea) 

^ the number of agreements between observers and 

t^e total number of observations* In t^is case, 0 was always 
20 since 5-minutc blocks were used. Agreements were based on 
t^e number of matched pairs — bot^ observers >-ad identical 
notations on t^e 1 5t^ second, t^r 30t^ second, t>-e U5th second, 
etc. T^e C|*s obtained for 24 training trials ranged from .41 
to 1.00 wit'- a mean coefficient of .77c 

Consistency Coefficient (Cj)* Tt would be illogical and impossible 
to calculate a Consistency Coefficient in t^e :nanner in w^ic^ t^e 
Cj was computed because w^ile we expect t^e gcMieral trends to be 
t<^e same, it would be presumptive to expect identical sequencing 
across occasions. Consequently, a different computational procedure 
was used to determine C2 t^nn t^at w^ic^ was employed for C « 



using t^e formula / \2 

^1 
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It is t^ere t^^at 1 would part company ...i t'^ Medley and Mitzel; t^^cy assume 
or at least i.ply t-.nt t^. observer is a flawless recording deviccfS^ 
unwarranted assumption fro. my perspective. T prefer to regard b as a 
consistency coefficient- to wi^at extent does t-^e observer Jepor? tC, 
same behaviors on different occasions, even tl^ougl^ bot'^ observers and 
bc^^avior may Vrjry. 
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1-0,2) 



(Lo) 

- tH' nuinber of paired observations , 0. = t^p frenuencv 
of category i on trial 1 and 0, j = t^^e frequency^or category i 
on trial 2. T^^e C^'s obtained across 24 training trials for eac»> 
observer ranr.ed from .51 to 1.00 wit^ a r^ean coefficient of .84. 

c. Reliability Coefficient (C3). Cor^putational ly, t^^e reliability 

coefficient is identical to t»^e consistency coefficient or stability 
coefficient; t^owever, rat»^er t»^an v^aving t»^e sar^e observer on two 
different occasions, t^e protocols of different observers are 
utilized. Tbe C3 's obtained during training ranged fror. .33 to 
.M witt^ a wean C3 of .72. The trials ised to compute t^e C 
were not t^e same trials used to obtain and data. 

Beginning in May, cac*^ teacber participating in tt^e study was observed 
on tbree separate occasions by eac»^ observer. Eac^ teac»^er knew a»^ead of 
time wKen tc . .-p, observer. TM>s, for eac». teac^^er t».ere were a 

grand total of 240 observations of student-tea c»^er interactions and 24 n 
(wKere n=tNe number of students-approximately 22- in t^c class) observa- 
rlons of student be»^aviors, 

3. Results . Tables XIV and XV present t^^e results of t^^ese observa- 
tions for t'^e teac'-er-student interactions and studert be»^aviors respec- 
tively. As Table XIV indicates, t^ere are pronounced differences among 
instructors wit> respect to tt^e interaction patterns. T^^ere are significant 
differences between t^^e Small House instructor and t^e control groups taken 

ti l ITa °"'a°'-''' dimensions or combinations (1-5 singly, 

li^^, 3 & 4, 2 & 4). Again, '^owever, it is evident t»^at i^e differences 
among t^e control group teachers equal or exceed some of t^^e differences 
between Small House and t»^e control teachers taken as a unit. T^e singular 

"wev' °" '\ v"" ~ °' teacher-initiated academic interaction^ ; 

however, as indicated earlier, t^e particular program implemented in S,.all 
House was an individualized program. 

It is also interesting to net e t^at only 80% of t'^e teac»^er-student 
interactions in Small House were academic in nature wMle in t^^e control 
classes t^e percentages ranged from 91.57. to 95.9%, T»^is difference is 
significant and constant, and tended to be relatively evenly distributed 
between social interactions and control -or i ented interactions. T^e 
difference in percent of control statements, w'^ile not significant, 
is in and of itself a pV-enomena wort*^ considering. It ^as been axiomatic 
t>^at If you offer new and unusu.-,l opportunities to students t»^ey will 
respond by behaving appropriately; r^ese results would not support t»^at 
particular contention. No reco,-,! was made of t^-e kinds of control 



22 
20 



TABLE XIV 

PERCENTAGES OF STUDENT-TEACHER INTERACTION MODES FOR FOUR MATHEMATICS CLASSES 

TYPE OF INTERACTION TEACHER A TEACHER B TEACHER C TEACHER D 
Teacher- initiated 

academic 40«7 67,2 85o0 55.2 
Student-initiated 

academic 39.5 24«2 7.0 40.7 
Teacher-ini tiated 

social 2.0 3.2 1.0 0.0 

Student-initiated 

social 5.2 0.0 0.0 1.2 

Teacher- initiated 

control 11,1 5.4 7.0 2.7 

TABLE XV 

PERCEOTAGES OF STUDENT BEWAVIORAL PATTERNS FOR FOUR MATHEMATICS CLASSES 

TYPE OF BEHAVIOR TEACHER A TEACHER B TEACHER C TEACHER D 

Traditional learning 58.4 57.5 59.2 78.2 

Nontradi tional learning 9.4 2.1 2.2 2.1 

Socializing 22.4 24.8 19.1 9.1 

Observing/ passive 

participation 1.8 5.8 .72 U4 

Daydreaming/withdrawal 8.0 9.8 18.8 8.5 
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(positive or nep,ntive) used. 



SoTOt^^ina t>at is not evident from t^e tables is t»B t t>>e cKi racteri st i c 
interaction patterns were ^^enerally >^ig>^ly stable from day to day and 
across observers^ As will be s^own later, t>>is too appears to be a function 
of t>^e particular toac>>er. Table XV suggests t^at student be^-aviors mig^t 
be independent of interaction patterns, flt^ a few notable exceptions, 
t^e percentages of students falling into eac^ of t^e categories were 
virtually identical for all classes. Again, wit>>in teac>>ers, t^ese per- 
centages were st.ible over time and across observers. 

Ty>c comparable data for t^e ot^er t^ree Small House teachers is presented 
in Tables XVI and XVII. On t^e average, t^ere are more student-initiated 
interactions and fewer teac>>er-ipitiatcd ones for t^^ese teacbers t>-an 
for t>-e ot>-ers. Also, t^ere tends to be significantly more teac>>er-student 
social (as opposed to academic) contacts in Small House. Conversely, 
contrary to expectation. Small House instructors made more frequent use 
of control statements and t>^e students' activities indicate far more non 
traditional learning instances t^an occur in t^e control groups. T^e 
regrettable lack of control classes in Science and Unified Studies makes 
it impossible to determine w^et^^er t^ese differences are attributable to 
differences in subject matter or teac^-er role. 

It is important to note t>-at t>-ese results are based on observations 
of required classe^i In Small House. Twus, t^^ey do not take into account 
t^e ot^er very broad activities and opportunities offered t>^e students. 
T>>is tact vns absolutely essential to minimize t^e differences between 
subject matter areas sine- most of t>>e electives offered were offered t^^roug^ 
Unified Studies. Also, t^e observations of nonrequired classes were far 
more difficult to code and less reliable. Nevert^^eless, many observations 
relating to t>-ese classes will be included in Oapter V. 

Following t^e recommendations of McGaw et al (1972) we can identify 
t^e variance components of t>>e observations by creating a breakdown 
similar to t»>at of Figure 1 (page 25). Using suc»^ a model it will be 
possible to identify t>>e extent to w^i cK teac^^er/student behavior patterns 
vary from day to day, >>ow muc>- observers tend to differ, and how muc>^ 
interaction t^^ere migKt be between observers and teachers. Applied to 
t^is particular problem, eac^ vector represents t^e frequencies appearing 
for eac^ observation period. It is t^en possible to do an Analysis of 
Variance, using vectors as entries (see Figure 2)» and ascertain t^e 
variance due to teac>>ers, observers and teac^^er-observer interactions 
by categories. 
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FIGURE 1 



MODEL FOR IDI-NTIFVINJ VARIANCE COMPONENrS OF OBSERVATION SCALES 

(GENERAL) 



Observer 1 



Observer 2 



Observer k 
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Sj = Situation 1, S^ ■ Situation n, a and T) are dbservations or frequency 
counts (for category scales) made by Observer k wH le observing Teacher i in 
Situation n. Ti.us, for example, if you ^ad 3 observers, 6 teachers, and eac^ 
observer observed eac^ teacher 3 times, you would v-ave 5A matrices, i.e., ^3 x 
values f or a o o • 1? • 
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FIGURE 2 

SAMPLE ANALYSIS OF VARIANCE SM/\LL HOUSE OBSERVATIONS 



SOURCE SUMS 


OF SQUARES df MEAN SQUARE F-RATIOS 


Observers 




10.6 

3c3 




ro.6~ 

3.3 


.33 
.31 






• 
• 

-5.1 


1 


• 
• 

5.1 


• 1 

U4 


Teachers 


1112o3 
• 

70. 5_ 


6 


i29o4' 
185.3 

• 
• 

,11. 7_ 


13.0 * 

19.1 * 
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Trac^^rs x Obs 


'124.2' 
231.6 

• 


6 


"20.7" 
38.5 
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.6 * 

4,0 

• 1 






12.' 8_ 




• 

2. 14 


• 1 

.4 


Error 


"944." 
266. 


28 


33.8 
9.5 

• 

.5.'5_ 




Total 

L 


3654. g" 
1612. 

• 
• 

242. _ 


41 







* Significant at .01 level 
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T»>o results presented in Figure 2 indicatG t^^at t>^cre are no main 
effects due to observer differences and t>^<nt t^ere are significant differences 
among teac'^ers for all categories wit'- t^e exception of teac^^er- ini t i ated 
control. But, t^erc is also a significant difference in t^e interaction 
betx,;een tcac»^ers and observers for t^e use of one category (student-initiated 
academic interactions). Thi^ suggests either t^at for one or two of t»>e 
Instructors certain be>^aviors were not stable over time or tJ-at in fact 
one of t»^e observers was biased in »^is coding for one or two teac^^ers. 
Since t»>e coding was a must coding system, this latter seems a more 
plausible explanation. 
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CHAPTER V 



NONSYSTtMATIC OBSERVATIONS 



to .Soiw °n , T . '° difficulties inherent in attempting 

to supply an adequate description of Small House from anytMng more tv^an 
a superficial structural perspective. T^e intent of tMs c»^apter is to 
not" it nJll -^l^^are relevant to tMs research but wMc. would 

Z Jl "^f °f t^e predpfinr-d categories, some of „M_k could 

be subsumed under tVe rubric of "confound v-.g variables- and some of wMc^ 
^ter'ai^r'f generalizations concerning some, area of concern. 

^^ l\ I following chapter) an effort will be made to integrate all 

\JT.^ Pf"^^''' ""'"^ ^'""^ °f coherent whole, discuss 

both the methodological flaws and the ramifications of this research and 
present the skeleton of an evaluation model. research, and 

All of the data in this chapter are based on voluminous notes taken 
?n IndeH ^''K f °' '^'^ by the observers; they are in no way 

intended to be presented as systematic or representative observatio ns and 
should not be construed as such, in some instances a single issue or inference 
made has been derived from a large number of observat io nsf at the same time^ 
times.°""""''^^ "^^ mentioned only once, though they happened twenty " 

A. SOME CONFOUNDING VARIABLES 

f,.H f ^f'^ll^r there were several cogenc variables which may have 

had a significant impact on the Small House program; some of these are 
directly related to the team's goals, others were functions of the team 

lllTrlV" T:''' ''''' "^"^ functions of outside influences. 

Regardless, they are issues that must be addressed. While it might be 

(Lriven i5ft7'°"'Jn'^,r''' ^^^^^^les an integral part of Small House 
inMMt ; in 'mi? '° '° =° specifying what they a re could 

inhibit replicability and general izabi 1 i ty. 

^' Administrative Support. That Small House was first implemented at 
Slauson was not entirely accidental. The school's administration was 
particularly committed to the concepts espoused in the Humaneness Report. 
The principal actively supported and encouraged the Small House team from 

for l^r.T.r year,without attempting to assume responsibility 

tor directing the program. 

One of the essential components of this support was the granting to the 
he?r';? designing their curriculum, handling 

their discipline problems, and working through a lot of technical details. 
This freedom to experiment" accorded the teachers in turn seemed to generate 
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within t^cm a tremendous nniount of i nvclvcmcnt« T^cv appeared willing to 
iv-ork twice as t>nrd as tl^ey previously ^^ad-aI.d ti^cy admitted to ot»^er^ 
instructors t1>nt it was more work-in order to keep tUc program runnings 
At times, l-Ke toac»^^rs wore spending ti'eir own money for some of t^^eir 
projects. TKc tcac>^crs acknowledge t'vnt two of ti^e' reasons t»^cy were 
willing to put forti' t'n^ effort were t'^e psyc»^ological atmospt^ere (at least 
from tKe supervisory level) tlvit was conducive to tt^eir ideas and t^e fact 
tl-at t^^is was tt-e:. program, not somct^^ing imposed from without. 

2. Selection RiaSc One ot»-er consideration t>-at migJ^t attenuate t>-e 
results is t^c selection biases involved— for bot^ t^e students and t^e 
teac»^ers. On t^e one Nand, all of t^e students in tl^e program were tl^ere 
because tt^eir parents-for whatever rcasons-t^ad decided t^at. an alternative 
educational experience was preferrable to t^e more traditional offerring 
available. Consequently, tt^ere is at least one, and possibly more, psyct^ological 
dimensions wt^icb would distinguisl. Small House from any easily obtained 
control group. 

On the ot^er hand, the teac»-ers tt-emselves were volunteers (p 1) wbo 
^ad been t^and-picked by t^e principal from t^^e larger group of volunteers. 
The fact that they had volunteered again suggests that they--along with the 
rest of the teachers who volunteered—were different in some respects, 
along some dimension. The fact that they were selected by the principal 
suggests that they fulfilled whatever criteria he was using more so 
than the other instructors; this only further complicates the issue. 

3' Team-Cluster Structu re; Side Effects . Although the concept of the 
planning team was an integral part of Small House as designed by the team, 
it also generated a number of side-effects which could not be considered 
as fundamentally generalizable. The planning team and the flexible modular 
scheduling made these things possible; however, the fact that they were 
conceived and executed was a function of a number of other variables. 

For example, several different groups were involved in operating a number 
of different programs for Small House students that the teachers themselves 
did not design or implement. These experiences, nevertheless, were offered 
as electives for students in Small House. One of the outcomes was that 
class size was often diminished considerably for students taking courses 
offered by Small House teachers. This extra help itself is a confounding 
variable. 

The cluster structure also seemed to have a significant impact on 
relations with parents. The teachers could plan events such as a picnic or 
pot-luck supper without having to do it alone. During the course of the 
year, several such events were sponsored. One of the effects was to obtain 
active support from parentr^ of Small House students, many of whom proved to 
be resources for Small House. This would have been impossible (or virtually 
so) without the team-cluster strucutre; neverthelss, the team-structure 
itself provided no guarantee that such would occur. 
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^' Ppcrat: i oruij_Mod i Pi cat ion p Since th\s was tl^e first ytar during 
wMcK Small llouso was in operation, t^erc was- a great tleal of experimenting 
and changing going on over t^e course of t^e year. In many respects t^e 
program at t»>c end of the year was radically different from w^at it was at 
t^e beginning of t^e ycnr„ Consequently, t^e program mig^t be conceived as 
a series of treatments wit), continuity of personne 1 , structure, and programs, 
but differing over time wit^ respect to one or more dimensions. T^e degree 
to wl^ic?. tT^esc differences affected individual .tudents would be a function 
of t^e particular student. For tl^ose students wit^ special needs, for example, 
V^o required electives" concept may T^ave radically altered t^eir psyc^^ological 
environment whereas ot^er students would ^ave been unaffected by ito 

^' T^e Novelty Syndrome, r^e evolution of any deviation from established 
practice-especially in education-tends to act as a rragnet for persons wit), 
abiding interest in educational reform. Small House was no exception. At 
one point in t ime, there were eighteen (18) different adults participating in 
some capacity- in t^^e operation of Small House. This included teachers, 
student teachers, C-390 students, parents, and others. This in and of itself 
would not necessarily be a particularly cogent variable except that the 
philosophy of the Small House team was to involve everyone to the maximum 
degree possible; people were permitted to come in and just observe, but 
were encouraged to do more than just watch. From their first day on the job, 
student teachers were pushed to get to work, encouraged to develop an elective 
or electives in accordance with their interests and to teach, c-390 students 
more often than not ended up by getting involved in some special project. It 
was very interesting to note that the teachers allowed their student-teachers 
and others the same freedom which had been granted to the team. 

This is not considered an exhaustive list of the possible confounding 
variables, simply an identification of the major ones. While it would be 
impossible to isolate the specific contribution of these factors in any 
statistical fashion (let alone their interaction), it is necessary to 
recognize that they constituted part of the treatment included in the term 
Small House. For example, if the freedom and support given the teachers 
provided the impetus for much of the teacher involvement, then this freedom 
and support would'be a necessary ingredient for other programs in the future- 
even though it is highly possibly, indeed probable, that a different set of 
four teachers would design a program that was radically different structurally 
and/or psychologically. This creates a tremendous paradox with respect to 
essential reproducibility, i.e., you cannot guarantee both the freedom and 
the structure (psychological and physical dimensions) of the program in 
any attempt to replicate with a different set of instructors. By imposing 
the structure, you remove the choice on the part of the instructors: by 
granting the freedom, you leave open t^e possibly selection of an alternate 
structure, 

B. SOME GENERAL OBSERVATIONS 

It would be easy to develop a rigorous program for studying seme 



30 

34 



phenomenon and execute it without every stopping to consider vhat one is 
not nicvnsurinf.^ There war; a grcnt deal more to Small House than what hns 
been presented by examining test scores or systematic observations; the 
constraints of the system plus the limited resources of the evaluators 
combined to reduce the capability to produce as thorough a study as was 
deslredo The data that follow are fundamentally anecdotal matrriais which 
highlight aspects or dimensions of Small House not evident in the cut-and- 
d.^ied typo of data presented in the first four chapters. Besides being 
of value for understanding some important ideas about Small House, they 
Identify topics which might be addressed by future evaluation studies, 

1. p hs ervati qn.s_ of Individual Students . One source of data ignored thus 
far IS that which was generated by observing individual students and changes 
in these students over the course of the year, Nonsystemati c observations 
of these students indicate radical changes in behavior between the beginning 
of the school year and the end of the year. The difficulty with interpreting 
such changes is that confounding variables mus^ be considered. There were 
other significant events (psychotherapy .behavior modification interventioi 
projects, etc) impinging on the lives of these students which could have 
effected these changes in and of themselves. All such cases were excluded 
from data presented below. 

. K. ^' .V^^""^ "^''^ Students in S^^all House who were designated as Class 
A Mentally Retarded individuals. In the early part of the year, there were 
some cruel remarks directed at these two; no such remarks were heard or 
observed after January and at no point was teacher intervention required. By 
the middle of the year (December- Janaury) these individuals were well integrated 
Into group activities. This was in sharp contrast to t he two Class A students 
observed in other classes who remained virtual isolates throughout the school 
year, 

b. There were three or four students who had had extremely poor atten- 
dance records throughout elementary school whose attendance at class was 
reguxar. Record keeping being what it is, precise figures are not available • 
to compare the figures among years. In the judgments of administrators and 
teachers, and according to their peers, there were significant differences 

in the attendance of these students, 

c. On at least one occasion, a student enticed to cut classe refused to 
do so, 

d. Six or seven students with poor academic records were observed on 
more than one occasion working on their own during break or during one of 
their electives on some area with vhich they were having difficulty. On 
some of those occasions they were soliciting assistance from a teacher or a 
peer. 

e. Even some of those students whose academic, attendance, and behavior 
records were particularly good demonstrated some marked changes— becomi ng 
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somcw^at more open nnd friendly over time, getting involved in different 
S°i>ar1nt"l r^M^orts!'' ''''' observations were substantiated 

nn I" '^^^'■'^^"^''^ -^"^11 "^>"'ber of students (4 or 5) wbo demonstrated absolutely 

no cKanges between September and June and wt.ose behavior warranted change^ 

One of t^ese students dropped out of the program (and tt.e public schools) aftrr 

one semsester; the otber four stayed for the entire school year, T^e crucial 

consideration bere is whether tbese kids would bave had any chance at all 

in a normal school program-wM ch is botb an unfair and an unanswerable 

quest! on© 

^. 2' Rap Sessions yith Students , Muc^ of our data came from informal 

nrn^M hI-^-'" J"" ^"'"^ °" ^"^^^^ T^ese discussions 

provide additional insight into the process of S^^all House as a treatment 
milieu. The basic findings of t».ese discussions could be summarized as 
follows: 



a. 



Ninty percent of t^e students in Sn,all House viewed it as an incredibly 
good tMng during t^e early stages. It „as different and exciting and as a 
group t^ey were getting a great deal of attenti( 
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b. Beginning in December, many of t^e Small House students began expressing 
the sentiment t^at t^ey were bored (more so t^an their counterparts in tradH^ 
t onal classes). A significant percentage b'^gan expressing a sentiment voiced 
ll^n%t tl uV!'" learning anything. Behavior problems became more 

common than they had been. This precipitated a change in the policies which 
the Small House T.am had towards discipline, an event which alienated some of 
the students even more. I feel t^at at this point in time the halo effect 
had worn off for both the students and the teachers. The general flatness 
and disenchantnvnnt that seemed to occur in Decmeber and January appeared to 

th.l ^''^r °" '"^^ P""' °f '^'^ teachers (who had been running 

themselves ragged . modification of the Small House policies, and an incredible 
emotional upheaval concerning the future of Small House. 

There was a noticeable change in the program beginning in late January 
or early February which „as reflected in .tudent behaviors, teacher behaviors 
and discussions with the students. According to the students, this renewed 
interest was a function of the fact t^at they were having more input into the 
selection of alternatives for electives and other changes that they saw the 
teachers making. Changes in the discipline policy were also cited. In any 
event, between February and June, the program developed a high degree of 
consistency even though the teachers continued to experiment with ideas. 

c. Discussions with the non Small House students elicited mixed reactions. 
Most admitted that they were envious of the opportuni ti tes open to the Small 
frrT^'ikr"?' P^^-^^^^^^^-^y ^^^^ things as the camping trip. However, approx- 
imately of those interviewed claimed that they voould not want to be in 
Small House for several reasons. One was tVat the kids in Small HqUsc were 
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»?osi?"r" ?t ^ V '-T "' "^''"y °f "^^^^ Sn,nll House were 

losers. It IS impossible to ascertain »>ow muc^ of t^is is sour Rrapes. 
l^ow mucK of u reflects t^e ideas of significant ot^^ers or How n,uc' oTit is 
a function of peer group pressurco 

f^nJ^lu ^'^^''^"^^^ ='"'''^1 """^ber of students in Small House w>^o maintained 
from tKe srnrt tK.t tHey were not particularly enthralled with the concept 
of Sina 1 House and w^o disliked it throughout. These students included 
individuals from all spectra-academical ly advanced as well as academically 
retarded, black and white, male and female. The one characteristic which 
they shared was a total lack of involvement in Small House activities. 
Most of them did not go on the camping trip or become involved in any of 
the other projects open to them even when encouraged to do so. 

e. In addition tD the aforementioned students, there was another 
sma 1 group who admitted that while it had been a lark for one year they 
would prefer a more normal eighth grade experience. Some of them admitted 
bluntly that they were afraid that they had not learned anything, other 
indications to the contrary, 

^' G'^"e^-al Classroom AtmosDherp. There was a qualitatively different 
atmosphere involved in the two settings. However- as in other areas - 
the difference among some of the control classes were as great as the 
differences between Small House and the control classes considered as a 
unit. Basically, the Small House classes were much more lively - including 
both positive and negative kinds of interactions. There seemed to be 
more laughing and smiling on the part of the Small House kids than generally 
was evident in the control classes. A word of caution is in order. Our 
experiences at Slauson indicate that tv-e psychological environment of a 
classroom is a highly variable factor, a function of many other factors 
ultimately dependent on the perceptions of the individual student and 
probably independ en t of t ^e teacher's behavior in the classroom as viewpH 
py an adult observer ! ' — 

Perhaps the most interesting observation about classroom climate was 
with respect to one of the control classes. The first term it was a hin^y 
routinized setup with minimum student fa rt i ci pa t ion and a higu degree of 
apathy and hostility - in the judgment of observers. That same class in 
the second term was radically different; there was a particularly strong- 
willed and hard-working student teacher who was granted permission to 
try out her ideas - with remarkable success. The instructor himself 
admitted that he could see and feel the difference in the class. Essen- 
tially they Kad gone to totally individualized instruction. There were 
fewer behavior problems, the kicfe were still learning, the noise level 
was higher out even the instructor was happier. However, after the student 
teacher s departure, he returned to the former mode of operation since the 
alternative was "too much trouble" to run by himself. 
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C. ErriMATION OF GOAL ACHIEVEMENT 



Scriven (1967) ).as made t».e crucial distinction between evaluation and 
estimation of goal achievement. And, w»^ile ow may not concur wMle-^eartedly 
with his granting of seeming omnipotence to the evaluator, this specification 
Of evaluation as more t^an estimation of goal achievement is essential for 
clarity. 

The Small House team had set themselves a series of goals prior to 
implementing their program. As part of thp evaluation process, it is 
legitimate to attempt to determine the extent to which they approximated 
those goals. Not all of these goals will be addressed; soire of them were 
stated in such ambiguous terms that attempts to assess them even peripherally 
were abandoned^ It is also important to remember that these judgments are 
subjective estimations based on data which „ere sometimes contradictory. 

^' Goal; Improved Rradine Ski 1 Is,. Given the limitaticns of the control 
group in this particular instance and the limi'ed reliability of the test, 
any answer to this particular question must be qualified. The Small House 
students seem to keep pace (using mean gain scores) with the control group: 
the gains demonstrated by the Small House students over time (mean gain in 
raw score) is equivalent to that of the national sample. Unfortunately, 
the reliability limitations and the entire "change score" problem prohibit 
making any effective judgments about individuals. Nevertheless, there 
were several other indications that for several students who needed it the 
most .this goal was effectively achieved. 

. ^ 2* Goal; Improved Math Skills, The larger control group inv) Ived in 
this case provides some measure of comfort and a little more faith in these 
results. Again, the gains demonstrated by the Small House students were 
comparable to these of t%e control group and equal to those expected on the 
bas.s of the national sample's mean gain. Again, the reliability question 
and the change score" problem qualify the affirnation of the achievement 
or this goal. 

^' S^Sli — Turn Students On. Some of the measures used to estimate 
this particular goal indluced absenteeism and tardiness-both in r-'lation to 
the control group and when using each student as his own control. This 
latter was done particular 1- for those students whose records indicated a 
high degree of absenteeis .ardiness in thp past. Other measures included 
the School Sentiment Index, discussions with students, and observation of 
pupil activities. (See Appendix ). The degree of success which the Small 
House team achieved for most of the students (80% or so) was hig»,. Tue 
frequency of positive instances of examples of this type of behavior seemed 
to increase over time for most of the students. There were a few students 
who literally never seemed to be interested in what was happening. 
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A. GonlL, Tn^er per so nal Growth ; In addition to t^ose items on the 

ItTilitTr-' f '"^"'"^ '° ^^^'^^'■^^ -'^^ included 

specific bt-Vaviors (soe Appendix ) dotnonstrated by t^e students. T»>ere 

observed. "''^'"''^ q^-^^tionnaires and son,e of the events 

One example of a significant event is tt.at of a girl w^o approac^ed 
one of t^e teachers asking „^at s>>e could do about one of t^er friends. Her 
friend was extremely biggoted; t>.e student wanted to remain friends wit^ 
tMs person, yr t had very strong feel'.ngs about bigotry. T^e two t^ings 
Mhy of note here is t^at t^e issue itself was enoug^ of a concern to 
the .student to cause r to raise t^( issue and secondly t^at she c^osp to 
do so with one of t^e teachers. 

Anot^er striking example in t^is domain centers around one of t^c black 
students in Small House. Originally, Ms only contacts wi t>> ^is peers 
were wit^ two of t^e ot»>er black students in Small House.. He was also very 
T^ost le towards almost all of the adults wit^ whom he came in contact. His 
participation in any group activity was virtually nil. AltHougH it took 
almost six months to alter this even minimally, by June there was a high 
degree of nteraction centering around this particular student, he had begun 
to actiyiely participate in many of his classes and was even observed smiling 
(something that had not happened at all in the first six months of obsa-vaHon 
on several occasions, 

..ur^T ff^ ^""^ dramatic two - of a number of events 

within Small House which reflect the attainment of this goal. Again, there 
was^some small number of students for whom this would not necessarily hold 

^' ^Sali Intraperso nal Growth/ Imp rovpH ^elf Image ; Originally, it 

was hoped to assess this particular variable using a Combination of psycho- 
^! iLi^"^?^" ^""^ behavioral observations. The latter were foregone due to 
the difficulty of arriving at specific behaviors that could be agreed upon as 
measures of sel f -concept ; thp former led to one of the more bizarre, but 
educational, experiences of the study. 

The Self Appraisal Inventory (developed at the Instructional Objectives 
Exchange at UCLA) was administered f:o both the experimental and control 
groups In September and again in Mgy. The original administration involved 
little or no difficulty, although 2-3 students ineach of the groups either 
refused to take the test or responded in such a fashion as to invalidate 
ItJlt^ In-^ ^^"."^ attempted to bive the posttest to Small House, however, 
more than 50Z of the students simply refused to respond, either in ^art of 
in whole. Many of them wrote notes on the answer sheets, after having 
answered none of the questions or responded only to those that they chose to 
answer. Some of the comments written are included below: 

. "You are too nosy*' (from four students) 
"Some of these questions are none of your business!" 

"I do not believe ehat any of this is anyone's business but my own and 
it is my choice to telly anyone, only if I want to." 

"I do not believe I have to answer that." 
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do not bavr to tell you any of this." 



I t t:^,nk ,t s any of your business to ask us these question.. 
My life is my business and nobody else's. You can look at my grades 
if you want to know „hotbcr I am a good student but I feel that I 
am under no obligation whatsoever to answer these questions." 

"Unless you are going to give each person „ho took this test some sort 

ii"Tj:::7ii^ " • » » -^es :e 

"You a re infringing on my right to privacy." 



disc^!^Il^of';hI'oL^i'"f' ^" opportunity to .it down and 
aiscu.s all of the parts of the study with a number of student- (30-407 of 

% nlr:4^"b:ut^"'\"'^^;^ ^° ^^^'^ ^^^^ a r^^saTlnvL fry. 

tio^and only ?aS obi 't.'S'' acceded to readily on the first administra- 

i dna oniy lacer obj-, -ted so strenusouslyo Almost to thp man 
students admitted th,-,t t„ey had felt pressured into aking thrt^sts-all o^ 
eVdid"u'to^":i,%V': 'T''' ^^^"^ - eiata^^in %::i".Ius .I.e.. 

^a^doml t e' rt^'a^JS T 'T'^' '''' ^" '''' ^^^^^^^^ 

develooed <=nn,P nil ~ answered always in a positive direction, or 

most par? thev Z,% l"""""""" ^^^fjegy. Fortunatly. they admitted tha for the 

the .tMMH ^ ? °^ tests seriously, including 

t' a udrs'ca' e'and ^h'^" '^f '''' '''' "'^^^ distinction between 

^^!nH ^^ ^ self-concept scale was that they could under- 

This entire experience has raised several issues concerning classroom 
research, some of «hich „ill be addressed in the fo 1 lowing secti ons Holvpr 

^an^^°gaTof 7 ^o^i^t'^^ °' ^ntrapersonal growth whic^is' fL'lore'reL": nr^ 
cnan a gain of 7 points on some arbitrary scale. 

^' ?;°al- G^^ater Student Invo v lvement in Lparning and F■ri., P.^^ nnd 

encourage each stude^^t'o re%rn:ibni ^ :r^;e\^L^n:r;r::th^^n the 
academic disciplines. Therefore, there were a^^ll^ite^Zb rTf required 
courses for each student. Even within these, students were frLuen^W 
offered options to do something else during that particular pSod f the 

~me'^\V:'teU'd?f d^^'J'^^^"^ "^^^^^^ °^ ^^^^ b^inTcover dV' 

who wcrrHoin^ did decide to impose one restriction. For those students 

who were doing especially poor work, some courses were required until th^r 
leZlrjorriZTV^'^ courses required included MaJh'Cb.^^tiiy' k lis. 
. ^f^p^l.^'^^::^^ -J very few 

:a?hrp;:ni?ri?:ni :T^t^:-ti~^ -^^-^-^^ 
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As mentioned earlier, even thouei M*. J Tk" constructlcn and design, 

very few of the stud .atl ^ecmed Jo nL'^f ? P°^*^y f"^"' t^e startf 

January letdown. After thrt thnViH K ^^^lously until the December- 
them, and In many cases took'the 1 «H ^^""^""g things that Interested 

example, some of'the stude^^ 8g ed'^^ffr'?!'^'^^"^'^" ^" ^"^^^ For 
member of the team agreed to do. Kinking f^ fu^ '^""^ elective, which a 
read or walk through a slmpL dr«,^« ^ students wanted to simply 

putting on a full produc Uon of .m' f "^ents, however, insisted n 

students assumed respons MHty m . The 

one in American History one in r °ther electi^J^Tl^cludi ng 

less academic fferings.* ^^°8raphy, and other areas as well as the 

^-n-HoSf^i^^^f^^ -rusts Of the 

w::^:e m^otLL^n^i^ def?si:n"ak?r^ 

own topiL and a^e 'and i'd s^^hi h ZTl?: T'^'''' '° ^^^"^"^ ^ -^f^^^r 
on the teacher to undertake a learn^nrivnor/ "''' '° """^ ^^P^"d 

domains. There were manv ^^«^«n! u ^''P"!^"" even in the traditioial 

enter « class anri^t^bo^u " h r work "[r.'' "'f" ^"'^^^'"^^ ^'^"^-•^^ -"^^ 

on a number of accasions, di ferent stude^?<,"° ^^^'^J"^'^" Present. Addltimally, 

or an elective period workinc on cl observed during break 

and even soliciting th^ as istanc of "e^'of'^J f ""^ '^^^^"^ difficuUy 

such things as write their n!n ^ 1 u . ^^^^"^ P""« Students did 

nonverbal^om^un^a ion process «L° ^'^^-^^P « -Itisense- . 

Issues for debate and theTdebaJed th m in '''''' developed 

was accomplished with .inimarguH'a^ce%:oVthe\e^Tr;. ""^'^ °^ ^'^^^ 

varyl^: Z^t^^l^ZlT^^^^^^^ -ese goals-to 

consideration. These Roals wr^. r?'. Indtoruces another relevant 

such, and to the cxtenfthat L °^ Small House and as 

"confounding variab es"' Sp 28 30)"'^f constitute additional 

(pp 28-30) affecting repl icabi li ty/generalizabi 1 i ty." 

D. THE NONREQUIRED CLASSES 

Tlu! list of elect Ives -■- ■• , , 

the alternatives avai lable to f7t*ud7^^ /"^^^^'^ 8"ss indicatim of 

18 only a partial list! The nonrlu red n/"' '^'^^^ 
reason than simply the diversify of i^nJ °^ '"'^'^"^'^ "'°re 

-oX ? -r^o- Hv^-b^"- 

-icany dif-rfnt 2-^tL-\%e^-r th^r^ti r%d-La-nol\^ 
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TV»c we 11 -de fined distinction between student and tenc^er— retained even 
in the Small House required classes — was considerably diminished in most 
of the electives; t^e atmosphere was more like an open classroom modeU 
In short, t^e differences between nonrequired classes in. Small House and 
required classes may Knve been greater tKin t>>o differences between Small 
House required classes and the control classes along a number of dimensions^ 

There are two substantial considerations focusing around the nonrequired 
classes that generate some concern and also ^ave implications for future 
such endeavors* The first is t^e question of required electiveso From the 
first, the teachers had offered among the electives courses designed to 
aid individuals in remediation of their basic skill deficits^ After the 
firts eight weeks or so of operating, the teachers felt that there were 
certain students who were making no progress and were in need of extra help 
who were not taking advantage of these options^ Their solution was co 
begin designating these students to take these electives (but never totally 
eliminating the options for this group) until they showed some improvement* 
The fact that they had to require students to take these electives went' 
against the grain of some of the team members — they considered it philo- 
sophically inconsistento For the most part the students involved accepted 
the decision as a reasonable one and some of them seemed grateful that the 
teachers had taken this action© 

The issue at stake is delineating the true philosophical tenets underlyh g 
the program. If you, are committed to some extent to granting students freedom 
of choice and are simultaneously concerned about his lack of preparation 
for the future and if you have a student unable to tolerate too many degrees 
of freedom, what takes priority? Is t^ere some course of action available 
whereby you can meet what at first glance appear to be contradictory goals? 

No answer will be attempted here. The important point is that such things 
must be addressed explicityly© From its inception the major stumbling block 
6f Small House was that though they had a set of goals, they had not established 
any priorities and could not generally conceive of a situation where t^ese 
goals might conflict* As a result, the development of Small House often 
floundered while t^ey worked through t^ese thirds operati onal ly. 

The second major issue centers on t^e dichotomy between the required 
classes and the nonrequired classes*. First, how can you classify an 
educational treatment that subsumes the gamut of treatments from laissez- 
faire to authoritarian, depending on the day and the ^our, as well as 
the teacher? Second, is it fair to ask students to accept this environmental 
schizophrenia, or are there developmental prerequi si t ies necessary for them 
to be capable of functioning in suc^ a setting? For some s tudents this 
seemed to be relatively easy; for others, impossible* 

This latter group of students tended to view the teachers as being 
inconsistent — a valid conclusion from a certain perspective* Why was 
it permissible to move freely around the room talking at one time and 
not another? If the teachers wanted them to enjoy school, why were 
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t^ey sti 1 exp£.cted to complete projects, demonstrate skills, and pass tests 
on occasions? W^y did t.oy Have to attend an elective just because ^ 
signed up for it? couldn't tKoy just go to t^e library or change 

their mind and go to a different elective or cut altogether? This eroup 
of students was small but significant. The fact that unlike most of the 
students «ho seemed able to grasp the nuances of the situation these 
students cou d not suggests that the ambiguity might be a little too much 
tor some students© 

^>- previously, the nonrequired classes were more than just non- 

traditiona content taught in a traditional fashion. They were the heart 
of the Sma 1 House operation and constitutrd an exceedingly complex set of 
of' h".,^ f '° "treatment" as defined by that term. Replication 
of the Small House experiment would be largely contingent on the ability 
to reconstruct the flavor and atmosphere of the nonrequired classes. 



E. ONE FINAL OBSERVATION 



It would be possible to continue generating additional anecdotes, 
observations, and other mater' ^1 relevant to this research. Tkg 
utility of so doing would be estionable. Nevertheless, there is one 
other piece of data which is .remendously significant. When the teachers 
announced that thc-y were running short of operating funds in the Spring and 
would not be able to fulfill their plans, the parents of the students con- 
tr buted between $1500 and $2000 to subsidize the Small House operation! 
This in a city which has defeated two millage elections in the last three 
years. In and of itself, this constitutes a penultimate evaluation of 
Small House. 
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CHAPTER VI 



Whpt ?<."'if ^ ^ re evant at tMs point In time to ask So „^at?" 

^nthe^lfd H ^ °' ^^'^"8 °f tMs Information? How can It be 
auesMoir 1^? '"'^^^^^'^^^ "^at does It mean? To attempt to answer these 
?esP«io^ %n!l.H/^*''f'"^ '° the historical context of educational 

research, Inc udlng the previous conceptions of currlcular evaluation. 

Jrleit'ttSn! ' '"'^ developments In philosophical 

oun^H^t/rJn"^"*' '"^'^^ °^ education Is not a new concept, Stevens (1967) 
^ITl I I °^ educational studies; the dghfyear study, one of the 

ZTrl IZrT, " examinations of education ever undertaken, wis Initiated 
ZZ lu \ /.^V^ of t».ese studies were predicated 

-nH .1 traditional psychological paradigm (wlt». Its underlying assumptions 
?nlff M °' Mstorlcally venerated, was technlcaUy 

Insufficient. Schutz (1969) puts It this way 

"Curriculum researchers have danced to the tune played by psycho- 
logical research methodology. The lyrics of psychological research 
met».odology have In turn been drawn for statistical analyses growing 
out of agronomy and biology. The disparity among t».e characteristics 
Glasfa^S).^^ ^ education was clearly noted by 

Hipr®^*^^! aut».ors J^ave also addressed this same Issue, though often from 
different perspectives: . trom 

••The stability ahd predictability of rows of corn have seduced us 
Into demanding (read assuming) stability and predictability In rows 
ot Children before we can examine and compare them properly .. . . 
We know next to nothing about the relationships between teachers 
attitudes, personality characteristics, and behavior. We have hardly 
touched on the relationships between teacher behaviors and learning 
in students.. . . These are difficult problems—com plicated, hard t n 

^ pprP fl Ch vlth trndltlonfll research ^.PthnHn^np ^oc, and well beyond 

the kind of problem we think we can deal with in research right now— 
but they are crucial Issues in education." ) (Allen, 1969) 

Tf*/ r occupation with satisfying the requirements 

of design and statistical models violates the Instructlonal'treatme^t 
and reduces the utility of the research to zero. Coversely. the lack of 
specificity Of treatment has often .ade the application of'OleganJ 
procedures a waste of time and money at best and a smokescreen at worst. 

T^e requirements for hn^^ -r" ci f Icatlon and desl.n must h° 

considered cnrefully ^o ...... re sults." fLk.r "p^.. 
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"Curriculum developers ^ave trnded to assume an isomorphism between 
tvelr interpretation of curriculum guides and instructional matf-rials 
and that of teachers and students^ * « „ However, word is leaking out 
tbat a number of c^eris^-ed curriculum beliefs are figments ot the 
researcher's theoretical imagination. Npngraded clas srooms appear to 
be indistinguishabl e from graded classro oms (Goodlad, 1968) ; self-inst ruc- 
tion in s c hools appears to be indistinguishable from teacher- in structi cn 
in schools, and endemic gaps are being identified "between emotionally- 
toned accounts of the ideology and the day-to-day reality in the life of 
innovative schools and classrooms" (S^it'^ and Keith, 1968, p 5). (Schutz, 359) 

"... Thp data would suggest that there really is no such thing 
as a BSCS curriculum presentation in the schools, , . „ Each 
teacher filters the materials through his own perceptions and to 
say that a student has been through the BSCS curriculum probably 
does not give as much specific information as the curriculum 
innovators might have hoped. (Gallagher, 1966, p 33) 

Traditional experimental psychology can justify the continued use of 
the agronomy-type of paradigm because, in their case, they can assume that 
the varaibles affecting the variable of interest are small in number, are 
being directly manipulated, or are controlled by random assignment of subjects. 
Additionally, they can specify the treatment so that it can be duplicated. 
This is generally not true foreducat i onal research. Although we can specify 
variability across classrooms, amon^ students, among teachers, and across 
socioeconomic levels (or any other dimension), we have no way at present of 
distinguishing relevant behaviors from irrelevant ones. In most cases, with 
educational research, we are dealing with nonrandom assignment of subjects, 
noncontrol (and often nonspeci f i cat i on) of treatment, and a large number of 
unknown variables, possibly having more unknown variables than subjects. 

What is needed, then, is a viable alternative strategy for guiding and 
channeling educational research, particularly those activities subsumed 
under the rubrics curriculum evaluation and evaluation research . we can 
no longer make simplisitic assumptions about treatments being equal except 
for the variable of interest; we can not make assumptions about random 
assignment of subjects or equality of treatment populations a priori ; we 
cannot justify labelling two instances of a treatmen' program with the same 
label without concomitantly ascertaining whether they ure in fact identical 
from every conceivable perspective! In a sense, we have to begin examining 
the schools as ecosystems with many components and evaluate results and 
programs with those specific components in mind. Our attitude here parallels 
that of Johnson (1970); 

" . . « The question here is not just what a program accomplished, 
but what caused it to do so. At a minimum we need a detailed descrip- 
tion of what the conditions were that yielded the given results. 
The practical problem is that of trying to describe what went into 



Cf Baker (1969) and Johnson (1970) for discussion of terminology and meaning. 
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the program In sufficient detail t^at t^e program ten be effectively 
reproduced. True, some social action programs may not be conceived 
of as sometMng to be "reproduced," but as single, one-sSot applica- 
tions. But for most purposes w»^en we speak of a new program of reading 
Instruction, or ai y pilot program of education, or social service, 
we are surely concerned vitally wlfr t».e question of wSetSer tSe program 
will Tiave the same effect w^en reintroduced In a new setting. The 
reproduclbllKy of t».e . program, and tSe extent to wMch suc». repro- 
ducibility is inherently built into it, is therefore of crucial 
importance in Interpreting the results of evaluative data on the out- 
comes produced by any program. ( p 20) 

The Issues addressed on pages 28-30 specifically focus on tMs problem of 
reproducing the program Intoto. 



EVALUATION MODELS: THE SECOND GENERATION 



Traditional models of evaluation are essentially obsolete. The newer 
models (systems models) are far more ecumenical about what .hould be considered 
data. Stake (1967) classified data into t^^re^ broad classes which Glass (1972) 
elaborates on. Stake refers to antecedents, transactions, and oU: comes, 
^tndfnt "^"n '^l^^^lfl'^ations are Intents (the planned-for environment 

student behaviors, etc) ;observatl pns (actual environment, student behaviors, etc); 

^a^h ^f^th'e^^oflow";: * ''''''' Observation 

Ant^^cdents-- 

1. Pupil rntry beV«avior and biograp>-l cal data 

2. Biographical and ot^^er data on t^e project personnel 

3. Pupil and staff expectations 

A. Parents expectations and wishes. 

5. Description of tV^e instructional environment. 

6. Description of t^e instructional, materials^^ 
7m Record of financial resources, 

8, The intended schedule and activities of tSe program 

9. Comparative data on the pupils and teachers of non-innovatlve classrooms, 

Trar^aact lons-- 
1# Actual sch dule of activities 

2. Tim^ sampling studlrs of how typical classroom hours are spent. 

3. The running record of the classroom (disturbances, significant 
deviations from intended activities, absences of staff and pupils, etc). 

Outcomes 

1. Pupil perofrmancr. and satisfactions both immediate and Img-rangc. 

2. Staff satisfactions and complaints both Irnmedlate and long-range 



* Glass, 1972, p 107. 
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3. Parrnt satisfactions and complaints. 

A. Pupil performance on cross-curricular obj ctlves. 

5. Staff mobi llty. 

6. Financial costsc 

7. Side effects of the program on nonpart Icl patlng pupils and staff. 

Now, Glass continues to say " AnytMng t>.at could concplvably be rPlated to 
the success of t».e program becom-s data for t^e evaluation** and that the only 
limitations under a systems model would be time and money (cf Scri ven,1967, 
P B3;. As Glass points up. Stakes model remains Incomplete-there are a myriad 
of seemingly crucial dimensions Ignored. 

A far more elaborate model, and perhaps the most comprehensive put forth 
yet, was suggested by Hammond (1972). He attends to the anthropological and 
sociological (Slndell, 1969; Harp and Richer, 1969) aspects of education 
in much grrater detail. His structure tor evaluation (Hammond, p 232) is 
highly reminiscent of Guilford's structure-of-i ntel lect' model. Hammond's 
3-dlmpnsional block Includes an instructional dimension, institutional dlmen- 
on, and a brhavioral dimension. Each of th^se Is further broken down, as 
indicated below: * 



INSTRUCTIONAL DIMENSION 



Organization 
Time 

Space--Vertlcal organization , Horizontal organization 
Content 

Methodology 

Teaching activities (16 categories) 

Types of interaction 

Learning theory ( 
Facilities 

Space, equipment, etc 

Cost 



INSTITOTIONAL DIMENSION 



Student (age, grade, sex, family variables, socio-economic 
input, physical hralth, mental health, achievement, ability, 
interests, relationship to innovation) 

Teacher (age sex, personaltiy characteristics, physical health, 
educational background and work experience, professional 
affiliations, nonprofessional affiliations, sociopconomic 
status of residence, degree of involvement in program, etCoj 

Administrators (same as te/icher) 

Specialists (same as teacher) 
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Psycf' OHIO tor 

Affective 

Cognitive 
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Family (degree of involvement wl t^ innovation, degree of planning 
general characteristics, size of family, socioeconomic status. ' 
pattern—nuclear or extended — income, residence, education 
of parents, siblings, mobility, social and professional affilia- 
tions. 

Community (geographical setting, historical development, popu- 
lation characteristics, populations size, density, change patterns, 
growth patterns, economic characteristics, sources and rangrs 
of individual incomes and tax bases, social characteristics, 
power structure, socioeconomic stratification 

BEHAVIORAL DIMENSION 
Psychomotor 
Affective 
Cogni tive 

nron^" ^^^^ition to tM s Structure, Pammond (p 235) goes on to elaborate a 
process, taking t>^e necessary step of establisMng t^at tMs structure is 
only a means of permitting an analysis of t^e interaction of t^e forces 
and Is independent of a process of evaluating. 

Hammond's model makes explicit some potentially significant factors 
w'^lcK could affect any attempt at educational innovation arti w»-ic'^ are 
Ignored (or unassumed unimportant) by ot»-er strategies or models. I am 
not suggesting, nor is Hammond, t^at we c»^ange one inappropriate model 
for anott^er; bot»- must be a part of any concprted effort to evaluate t^e 
c»^anges affecting and effected by an educational treatment or innovation. 

^M^rr^ T are left implicit in tt^e model of Hammond's 

r^a?n,t IttP^nM r''^ ^'^^'"P^^' B""^*^ (1971) cautions 

of^Ko . ? T^^ ^" classroom" ori-^ntation unless t»-e ideologies 

of t^ose involved in tt-e c»-ange are consistent wi t*^ t»-e c-^ange. It is my 

TullTnn f ' ^"''"^'^ °' '"^"y ^"'^'"P'^^ '° 1""°^-'- "e a direct 

function of a nonpM losopMcal or apMlosopMcal orientation on t>-e pirt of 

t^rnnn ^"^'^^^'^^ attempts. I t^avo witnessed several cases w'^ere 

t^v open classroom- process was simply legislated on to unsuspecting and 
often unwilling participants. TMs report itself falls s'^ort on failing to 
acMeve some index of pMlosopMcal orientations of t»-e different instructors. 

From tt^is on point (pMlosopMcal beliefs of teac^-^rs) we could generate 
an entire series of additional factors and interactions wMcrcou" affect 
t^e innovative process. For example : 
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Wt-at »>apppns if t»>ere is dlssonnance botween t»>e p^^ilosop»-ies 
of t^e teachers and t^osr of administrators, particularly t^e 
principal? 

W^at happens if t^-er^ is dissonance between t^o p»^i losop»^ies of 
practitioners (teac»^ers and principals) and t»^os of t^e school 
board (and/or community)? 

Is a teact-er's p>^ilosop».y as measured on paper consistent wit>^ Ker 
be^^avior? 

TKese questions can be added to t»^osp t»^at are already present in t»^e model 
developed by Pammond, 

Is ct^angP possible in a ^ig^ly transient community? If in^Mtut-d 
will it be maintained? 

VT'at is thp relatlonsMp between t^e psyc>^ologl cal environment of t^^e 
^ome and a student's c^a:lty to adapt in a particular sc»^ool environment? 

Is continuing eff-ctlvp c^^ange dependent upon a "strong man"?" 

V^at arp t^^e relations»^ips among fac^^ers' personalities, administra- 
tors' personalities and ffective continued change? 

VP'at kind of psyc'-ological environment are t^e teacher's working in" 
Wrrp tt^ey instrumental in planning f-e c»^ange? Were f-ey given t^e 
freedom (eaC- and every one of f-em) to design t'^eir onw programs? 
Do t»^ey all feel f-at t^e principal and/or superintendent supports 
t^em regardeless of w^-ef-er '-e agrees Hit^ t^em or rot? 



T»>ere i g one far-reacMng implication to all of t'i s and to t^^P question 
of an effective model for educational 'valuation anr' rcsearc'". Muc»^ of w^^at 
was done in t»^e past may. in fad:, br •. valid. - useful, but invalid. Still 
anot^-er, and one of far greater import .ncerns t^c future. Specifically. 

(19677? 83)t ^''^"^ '° '""^ ^ '^^^^ ^ "'^^ S"*^^" 

" . , . 'ft^e educational profes^ ]• ^'.ffoilng from a completely 
InappvoprJate conception of tK^ ?cr- -lo ^or educational research. 
To dnvelop a new rocket or aut en..' , v?ry exponsive piece of 
business, despite t»^o extreme con .• c;f tSr properties of p»^ysical 
materials. W^en we are dcall.ig v. .. ..cVing instrument, suc^ as 

a new curriculum or classroom proocdur--. .. i.t), ,ts extreme dependence 
upon extremely variable operators and re.' .^pp-s, we must expect 
considerably more expense. T^r social pa," •"f is enormously more im- 
portant , , . Jhp Pducariontu profession ^t, a wV,ole ^as a primary 
obligation to recognizp ii e d-Tficulty of y.^ci curriculum development 
wit»^ Its essential concomi tnnt: , pvaluation, a,,d to b^gln a unified 



attack on t*^e problero. of financing t*^e kind of improvement t^^at may 
^elp us towards tt^P goal of a few million enligt^tened citizens on t»^e • 
earts s surface, evan at t»^e expense ot one on t^e surface of Mars. 

In addition to t^e financial implications, Harranond's model t^as tremendous 
implications for mett^odo logies. T^^ simplistic, alleged objective type of 

P^P"'' P""^*^ 8°"^- Suct^ ^ '""del demands a 

multlmet^od approach particularly one wMc*^ utilizes naturalistic and systematic 
observation and ot»-pr clinical tec^^niques. 
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CHAPTER VII 



at a'^unlor '1!'"'^^''? '° ^° '^^^"^'^ P"?'^^'" Initiated 

at a junior MgK school, covering a period of one academic year; additionally 

le 6lLnirr JTu '^I'^'^'T''' °' ^'^^^'^^"^ approaches to evaluation „ou 
^dJ^M y "^^-^^^ ^'^^ =^^^"1 key points concerning 

on br °nv ir:;"' '""^'^ addressed; some ot t»^ese we »^ave touched 

on^brlefly In t»^e preceding pages. ot»^ers will be mentioned for t»^e first 
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a, 



b. 



c. 



d. 



f. 



If It Is at all possible, it is reasonable to attempt to ac>'ieve t^^e 
same degree of control as psyc^-ological experiments via random assign- 
ment, etc. If t>'e groups are already establ 1 s>'ed, t>'Pn t^^e "best" 
tvat we can possibly do is attempt to demonstrate t»'at-as best as 
^pr.pln'^, "'"^^ measure. .t»^e groups are equal along 

Soiever ' .I' '^P^^"^' ^" '"^'^ particular researC. 

k! ? ^''°"P' "'^^^^'^ significantly in any respect . tMs must 

^tui n V : ^'^^'"Ple. in tMs research all of the Small House 

students Vad parents w^o were willing to volunteer; t^ere is no way 
of equating for tMs "^ome climate" in any reasonable control group. 

Evaluation studies should cover a period longer t^an one year. It 
is impossible to rely on a determination based on one-year trial 
simply because it virtually takes one year to establish a program, 
iron out thp bugs, and get rid of the halo effects. s • . 

T».ere are at least two units of analysis in educational studles-- 

the class, and the Individuals. To ignore either one is to jeopardize 

the value of the study. Particular attention should be paid to 

% C'fo^f"'"' Interactions (where aptitude is broadly defined) 
(Messick.1972; Bracht. 1969). 

The kind of data gathered and the data-gathpr Ing processes must 
both be expanded far from what thpy ^ave been in the past. 

Perhaps the greatest task confronting educational research and evaluation 
is to determine which variables are irrelevant to the educational 
process an) which are relevant and what Interactions exist among 
the relevant ones. 

frl"^nin? '"J^'^uf ™^ r^""^" combining cri terl ai Judgments. and costs 
in a complrx fashion. For example, the distinction was mad- in this 
paper between estimation of goal achlrvement and evaluation, the former 

hai dpi;''' °' ''''''' ^' ^"PP"'"^ '"^^^ ^"^^ Students 

had demonstrated a significantly more positive attitude hut f^lr gains 

In reading and Hiflt*- ^-ad been significantly less th«n the control gLp? 
Jesearc^! ' Integrated into ^valuation 
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f.pp"nf ""'"T ""P"^^" °^ developing only aftrr it ^^ad broken 

free of t^e influence of Galnn. \^iU- Ke admittedly did much to advance tT.r 
Tnlloli '"'tr''"- ^'!'" eventually became a stumbling block to its continued 
IIJ Educational res.arc>^ and evaluation is in a position similar to 

I "f^^e^*^^"^; benefits t^at ^ave accrued from t^e Psyc»^ological model 

^ave wedded tKat model to education and precluded furt»^er development. Unless 
^nrJL^ T ' simplistic model, educational researc»^ and evaluatbn 

will become fruitless excercises and »^ypertropV^ied ritualism. 
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